diff --git a/HLog.md b/HLog.md new file mode 100644 index 0000000..6a50bfe --- /dev/null +++ b/HLog.md @@ -0,0 +1,476 @@ +# HLog - Structured Logging for Go + +**Version:** v0.10.1-hlogdoc + +HLog is a structured logging package built on top of [zerolog](https://github.com/rs/zerolog), providing simple configuration, flexible output options, and automatic log file management for Go applications. + +## Installation + +```bash +go get git.haelnorr.com/h/golib/hlog +``` + +## Key Concepts and Features + +HLog provides the following key features: + +- **Multiple Output Modes**: Log to console, file, or both simultaneously +- **Configurable Log Levels**: Support for trace, debug, info, warn, error, fatal, and panic levels +- **Environment-Based Configuration**: Easy setup using environment variables +- **Programmatic Configuration**: Full control via Config struct when needed +- **Automatic File Management**: Handles log file creation and management +- **High Performance**: Built on zerolog for minimal overhead +- **Error Stack Traces**: Automatic stack trace marshaling for errors +- **Flexible File Modes**: Choose between append or overwrite modes + +## Quick Start + +The simplest way to get started with HLog is using environment variables: + +```bash +# Set environment variables +export LOG_LEVEL=info +export LOG_OUTPUT=console +``` + +```go +package main + +import ( + "log" + "os" + + "git.haelnorr.com/h/golib/hlog" +) + +func main() { + // Load configuration from environment + cfg, err := hlog.ConfigFromEnv() + if err != nil { + log.Fatal(err) + } + + // Create logger + logger, err := hlog.NewLogger(cfg, os.Stdout) + if err != nil { + log.Fatal(err) + } + defer logger.CloseLogFile() + + // Start logging + logger.Info().Msg("Application started") +} +``` + +## Configuration + +HLog can be configured in two ways: via environment variables or programmatically. + +### Using ConfigFromEnv (Recommended) + +The `ConfigFromEnv()` function reads configuration from environment variables: + +```go +cfg, err := hlog.ConfigFromEnv() +if err != nil { + log.Fatal(err) +} + +logger, err := hlog.NewLogger(cfg, os.Stdout) +if err != nil { + log.Fatal(err) +} +defer logger.CloseLogFile() +``` + +### Environment Variables + +| Variable | Description | Valid Values | Default | Required | +|----------|-------------|--------------|---------|----------| +| `LOG_LEVEL` | Log level for filtering messages | trace, debug, info, warn, error, fatal, panic | info | No | +| `LOG_OUTPUT` | Output destination | console, file, both | console | No | +| `LOG_DIR` | Directory path for log files | Any valid directory path | - | Yes (when LOG_OUTPUT is "file" or "both") | +| `LOG_FILE_NAME` | Name of the log file | Any valid filename | - | Yes (when LOG_OUTPUT is "file" or "both") | +| `LOG_APPEND` | Append to existing log file | true, false | true | No | + +### Programmatic Configuration + +For more control, create a `Config` struct directly: + +```go +cfg := &hlog.Config{ + LogLevel: hlog.InfoLevel, + LogOutput: "both", + LogDir: "/var/log/myapp", + LogFileName: "application.log", + LogAppend: true, +} + +logger, err := hlog.NewLogger(cfg, os.Stdout) +if err != nil { + log.Fatal(err) +} +defer logger.CloseLogFile() +``` + +## Log Levels + +HLog supports seven log levels, from most to least verbose: + +| Level | Description | Use Case | +|-------|-------------|----------| +| `trace` | Most verbose | Very detailed debugging, function entry/exit | +| `debug` | Debug information | Detailed debugging during development | +| `info` | Informational | General application events | +| `warn` | Warnings | Potentially harmful situations | +| `error` | Errors | Error events that might still allow the app to continue | +| `fatal` | Fatal errors | Severe errors that will cause the application to exit | +| `panic` | Panic | Panic-level errors that will panic the application | + +### Setting Log Levels + +Via environment: +```bash +export LOG_LEVEL=debug +``` + +Programmatically: +```go +cfg := &hlog.Config{ + LogLevel: hlog.DebugLevel, + // ... other config +} +``` + +### Logging at Different Levels + +```go +logger.Trace().Msg("Entering function") +logger.Debug().Str("user_id", "123").Msg("User details loaded") +logger.Info().Msg("Server started on port 8080") +logger.Warn().Msg("Cache miss, fetching from database") +logger.Error().Err(err).Msg("Failed to connect to database") +logger.Fatal().Msg("Cannot start: configuration file missing") +logger.Panic().Msg("Unrecoverable error occurred") +``` + +## Output Modes + +HLog supports three output modes: + +### Console Output + +Logs are written to the provided `io.Writer` (typically `os.Stdout` or `os.Stderr`) with human-readable formatting: + +```bash +export LOG_OUTPUT=console +``` + +```go +logger, err := hlog.NewLogger(cfg, os.Stdout) +``` + +### File Output + +Logs are written to a file in the specified directory: + +```bash +export LOG_OUTPUT=file +export LOG_DIR=/var/log/myapp +export LOG_FILE_NAME=server.log +``` + +```go +// Note: io.Writer can be nil when using file-only output +logger, err := hlog.NewLogger(cfg, nil) +``` + +### Both (Console + File) + +Logs are written to both console and file simultaneously: + +```bash +export LOG_OUTPUT=both +export LOG_DIR=/var/log/myapp +export LOG_FILE_NAME=server.log +``` + +```go +logger, err := hlog.NewLogger(cfg, os.Stdout) +``` + +## File Management + +### Append Mode + +By default, HLog appends to existing log files, preserving logs across application restarts: + +```bash +export LOG_APPEND=true +``` + +### Overwrite Mode + +Set `LOG_APPEND=false` to truncate the log file on each application start: + +```bash +export LOG_APPEND=false +``` + +### Closing Log Files + +Always close log files when shutting down to ensure all buffered logs are flushed: + +```go +defer logger.CloseLogFile() +``` + +Or handle it explicitly: + +```go +if err := logger.CloseLogFile(); err != nil { + log.Printf("Error closing log file: %v", err) +} +``` + +## Structured Logging + +HLog leverages zerolog's structured logging capabilities: + +```go +// Simple message +logger.Info().Msg("User logged in") + +// With fields +logger.Info(). + Str("username", "john_doe"). + Int("user_id", 12345). + Msg("User logged in") + +// With error +logger.Error(). + Err(err). + Str("operation", "database_query"). + Msg("Operation failed") + +// Nested objects +logger.Info(). + Str("username", "john_doe"). + Dict("metadata", zerolog.Dict(). + Str("ip", "192.168.1.1"). + Int("port", 8080), + ). + Msg("Connection established") +``` + +### Field Types + +HLog supports all zerolog field types: + +- `Str(key, value)` - String +- `Int(key, value)` - Integer +- `Float64(key, value)` - Float +- `Bool(key, value)` - Boolean +- `Err(err)` - Error with stack trace +- `Time(key, value)` - Timestamp +- `Dur(key, duration)` - Duration +- `Dict(key, dict)` - Nested object + +## Advanced Usage + +### Custom Writers + +You can use any `io.Writer` for console output: + +```go +// Write to a buffer +var buf bytes.Buffer +logger, err := hlog.NewLogger(cfg, &buf) + +// Write to multiple destinations +multiWriter := io.MultiWriter(os.Stdout, &buf) +logger, err := hlog.NewLogger(cfg, multiWriter) +``` + +### Context Loggers + +Create sub-loggers with additional context: + +```go +// Create a logger with request context +requestLogger := logger.With(). + Str("request_id", requestID). + Str("user_id", userID). + Logger() + +requestLogger.Info().Msg("Processing request") +``` + +### Conditional Logging + +Check if a log level is enabled before expensive operations: + +```go +if logger.Debug().Enabled() { + // Only compute expensive debug info if debug logging is enabled + debugInfo := computeExpensiveDebugInfo() + logger.Debug().Interface("debug_info", debugInfo).Msg("Debug information") +} +``` + +## Integration + +HLog is designed to work seamlessly with other golib packages: + +### With env Package + +HLog uses the [env](https://git.haelnorr.com/h/golib/env) package for environment variable parsing in `ConfigFromEnv()`. The env package provides type-safe environment variable access with defaults. + +### With hws/hwsauth + +HLog integrates directly with the [hws](https://git.haelnorr.com/h/golib/hws) web server framework. The hws package provides an `AddLogger` function that accepts an `hlog.Logger` for structured logging throughout your web application: + +```go +import ( + "log" + "os" + + "git.haelnorr.com/h/golib/hlog" + "git.haelnorr.com/h/golib/hws" +) + +// Configure logger +cfg := &hlog.Config{ + LogLevel: hlog.InfoLevel, + LogOutput: "both", + LogDir: "/var/log/myapp", + LogFileName: "server.log", +} + +logger, err := hlog.NewLogger(cfg, os.Stdout) +if err != nil { + log.Fatal(err) +} +defer logger.CloseLogFile() + +// Create HWS server configuration +hwsConfig := &hws.Config{ + // ... your hws config +} + +server, err := hws.NewServer(hwsConfig) +if err != nil { + log.Fatal(err) +} + +// Add the logger to the server +server.AddLogger(logger) + +// The server will now use hlog for all logging +logger.Info().Msg("Starting web server") +server.Start() +``` + +The hws framework will automatically use the provided logger for request logging, error logging, and other server events, ensuring consistent structured logging throughout your application. + +## Best Practices + +1. **Always close log files**: Use `defer logger.CloseLogFile()` immediately after creating the logger to ensure logs are flushed on shutdown. + +2. **Use structured logging**: Add context fields instead of formatting strings: + ```go + // Good + logger.Info().Str("user", username).Int("age", age).Msg("User created") + + // Avoid + logger.Info().Msgf("User created: %s, age: %d", username, age) + ``` + +3. **Choose appropriate log levels**: Don't log everything at `Info` level. Use `Debug` for development details and `Trace` for very verbose output. + +4. **Use ConfigFromEnv in production**: Environment-based configuration makes it easy to change logging behavior without code changes. + +5. **Add context to loggers**: Create sub-loggers with request-specific context rather than adding the same fields to every log call. + +6. **Avoid logging sensitive data**: Never log passwords, tokens, or other sensitive information. + +7. **Use append mode in production**: Set `LOG_APPEND=true` to preserve logs across restarts for better debugging and auditing. + +8. **Check log level before expensive operations**: Use `logger.Level().Enabled()` to avoid computing log messages that won't be written. + +9. **Centralize logger creation**: Create your logger once at application startup and pass it through your application (dependency injection or context). + +10. **Use appropriate output modes**: + - Development: `console` for immediate feedback + - Production: `both` for real-time monitoring and persistent logs + - Services: `file` when logs are collected by external tools + +## Troubleshooting + +### "LOG_DIR must be set when LOG_OUTPUT is 'file' or 'both'" + +This error occurs when file logging is enabled but no directory is specified. Set the `LOG_DIR` environment variable: + +```bash +export LOG_DIR=/var/log/myapp +``` + +### "LOG_FILE_NAME must be set when LOG_OUTPUT is 'file' or 'both'" + +This error occurs when file logging is enabled but no filename is specified. Set the `LOG_FILE_NAME` environment variable: + +```bash +export LOG_FILE_NAME=server.log +``` + +### "Invalid log level specified" + +Valid log levels are: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`. Check your `LOG_LEVEL` environment variable: + +```bash +export LOG_LEVEL=info +``` + +### "Invalid LOG_OUTPUT" + +Valid output modes are: `console`, `file`, `both`. Check your `LOG_OUTPUT` environment variable: + +```bash +export LOG_OUTPUT=console +``` + +### Log file permission errors + +Ensure the application has write permissions to the log directory: + +```bash +mkdir -p /var/log/myapp +chmod 755 /var/log/myapp +``` + +### Logs not appearing + +1. Check that your log level includes the messages you're trying to log +2. Ensure you're not calling `logger.CloseLogFile()` before logging +3. Verify that stdout/stderr is not being redirected elsewhere +4. For file logging, check that the file exists and is being written to + +### Logs not flushed on crash + +When a program crashes or is killed, buffered logs may not be written. Consider: +- Using `both` output mode to see logs in real-time on console +- Calling `logger.CloseLogFile()` explicitly before risky operations +- Using `defer logger.CloseLogFile()` to ensure cleanup on normal exits + +## See Also + +- [env](https://git.haelnorr.com/h/golib/env) - Environment variable helpers +- [hws](https://git.haelnorr.com/h/golib/hws) - HTTP web server framework +- [hwsauth](https://git.haelnorr.com/h/golib/hwsauth) - Authentication middleware + +## Links + +- [GoDoc API Documentation](https://pkg.go.dev/git.haelnorr.com/h/golib/hlog) +- [Source Code](https://git.haelnorr.com/h/golib/hlog) +- [Issue Tracker](https://git.haelnorr.com/h/golib/issues) +- [Zerolog Documentation](https://github.com/rs/zerolog) diff --git a/Home.md b/Home.md index 64f7a6a..402ccd5 100644 --- a/Home.md +++ b/Home.md @@ -4,6 +4,9 @@ Welcome to the golib documentation wiki. This wiki provides comprehensive docume ## Modules +### [HLog](HLog.md) +Structured logging package built on top of zerolog. Provides simple configuration via environment variables, flexible output options (console, file, or both), and automatic log file management with high performance. + ### [HWS](HWS.md) H Web Server - A lightweight, opinionated HTTP web server framework for Go built on top of the standard library. Features Go 1.22+ routing patterns, built-in middleware, structured error handling, and production-ready defaults.