oslstats/AGENTS.md

# AGENTS.md - Developer Guide for oslstats

This document provides guidelines for AI coding agents and developers working on the oslstats codebase.

## Project Overview

**Module**: `git.haelnorr.com/h/oslstats`  
**Language**: Go 1.25.5  
**Architecture**: Web application with Discord OAuth, PostgreSQL database, templ templates  
**Key Technologies**: Bun ORM, templ, TailwindCSS, custom golib libraries

## Build, Test, and Development Commands

### Building
```bash
# Full production build (tailwind → templ → go generate → go build)
make build

# Build and run
make run

# Clean build artifacts
make clean
```

### Development Mode
```bash
# Watch mode with hot reload (templ, air, tailwindcss in parallel)
make dev

# Development server runs on:
# - Proxy: http://localhost:3000 (use this)
# - App: http://localhost:3333 (internal)
```

### Testing
```bash
# Run all tests
go test ./...

# Run tests for a specific package
go test ./pkg/oauth

# Run a single test function
go test ./pkg/oauth -run TestGenerateState_Success

# Run tests with verbose output
go test -v ./pkg/oauth

# Run tests with coverage
go test -cover ./...
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out
```

### Database Migrations

**oslstats uses Bun's migration framework for safe, incremental schema changes.**

#### Quick Reference

```bash
# Show migration status
make migrate-status

# Preview pending migrations (dry-run)
make migrate-dry-run

# Run pending migrations (with automatic backup)
make migrate

# Rollback last migration group
make migrate-rollback

# Create new migration
make migrate-create NAME=add_email_to_users

# Dev: Run migrations without backup (faster)
make migrate-no-backup

# Dev: Reset database (DESTRUCTIVE - deletes all data)
make reset-db
```

#### Creating a New Migration

**Example: Adding an email field to users table**

1. **Generate migration file:**
   ```bash
   make migrate-create NAME=add_email_to_users
   ```
   Creates: `cmd/oslstats/migrations/20250124150030_add_email_to_users.go`

2. **Edit the migration file:**
   ```go
   package migrations
   
   import (
       "context"
       "github.com/uptrace/bun"
   )
   
   func init() {
       Migrations.MustRegister(
           // UP: Add email column
           func(ctx context.Context, db *bun.DB) error {
               _, err := db.ExecContext(ctx, 
                   "ALTER TABLE users ADD COLUMN email VARCHAR(255)")
               return err
           },
           // DOWN: Remove email column (for rollback)
           func(ctx context.Context, db *bun.DB) error {
               _, err := db.ExecContext(ctx, 
                   "ALTER TABLE users DROP COLUMN email")
               return err
           },
       )
   }
   ```

3. **Update the model** (`internal/db/user.go`):
   ```go
   type User struct {
       bun.BaseModel `bun:"table:users,alias:u"`
       
       ID        int    `bun:"id,pk,autoincrement"`
       Username  string `bun:"username,unique"`
       Email     string `bun:"email"`          // NEW FIELD
       CreatedAt int64  `bun:"created_at"`
       DiscordID string `bun:"discord_id,unique"`
   }
   ```

4. **Preview the migration (optional):**
   ```bash
   make migrate-dry-run
   ```
   Output:
   ```
   [INFO] Pending migrations (dry-run):
     📋 20250124150030_add_email_to_users
   [INFO] Would migrate to group 2
   ```

5. **Apply the migration:**
   ```bash
   make migrate
   ```
   Output:
   ```
   [INFO] Step 1/5: Validating migrations...
   [INFO] Migration validation passed ✓
   [INFO] Step 2/5: Checking for pending migrations...
   [INFO] Step 3/5: Creating backup...
   [INFO] Backup created: backups/20250124_150145_pre_migration.sql.gz (2.3 MB)
   [INFO] Step 4/5: Acquiring migration lock...
   [INFO] Migration lock acquired
   [INFO] Step 5/5: Applying migrations...
   [INFO] Migrated to group 2
     ✅ 20250124150030_add_email_to_users
   [INFO] Migration lock released
   ```

#### Migration Patterns

**Create Table:**
```go
// UP
_, err := db.NewCreateTable().
    Model((*db.Game)(nil)).
    Exec(ctx)
return err

// DOWN
_, err := db.NewDropTable().
    Model((*db.Game)(nil)).
    IfExists().
    Exec(ctx)
return err
```

**Add Column:**
```go
// UP
_, err := db.ExecContext(ctx, 
    "ALTER TABLE users ADD COLUMN email VARCHAR(255)")
return err

// DOWN
_, err := db.ExecContext(ctx, 
    "ALTER TABLE users DROP COLUMN email")
return err
```

**Create Index:**
```go
// UP
_, err := db.NewCreateIndex().
    Model((*db.User)(nil)).
    Index("idx_username").
    Column("username").
    Exec(ctx)
return err

// DOWN
_, err := db.ExecContext(ctx, 
    "DROP INDEX IF EXISTS idx_username")
return err
```

**Data Migration:**
```go
// UP
_, err := db.NewUpdate().
    Model((*db.User)(nil)).
    Set("status = ?", "active").
    Where("status IS NULL").
    Exec(ctx)
return err

// DOWN (if reversible)
_, err := db.NewUpdate().
    Model((*db.User)(nil)).
    Set("status = NULL").
    Where("status = ?", "active").
    Exec(ctx)
return err
```

#### Safety Features

✅ **Automatic Backups**
- Compressed with gzip (saves ~80% space)
- Created before every migration and rollback
- Stored in `backups/` directory
- Retention policy keeps last 10 backups
- Graceful fallback if `pg_dump` not installed

✅ **Migration Locking**
- PostgreSQL advisory locks prevent concurrent migrations
- 5-minute timeout prevents hung locks
- Safe for multi-instance deployments

✅ **Validation**
- Pre-migration `go build` check ensures migrations compile
- Dry-run mode previews changes without applying
- Status command shows exactly what's applied

✅ **Rollback Support**
- Every migration must have a DOWN function
- Rollbacks are grouped (undo last batch together)
- Automatic backup before rollback

#### Environment Variables

```bash
# Backup directory (default: backups)
DB_BACKUP_DIR=backups

# Number of backups to keep (default: 10)
DB_BACKUP_RETENTION=10
```

#### Troubleshooting

**"pg_dump not found"**
- Migrations will still run, but backups will be skipped
- Install PostgreSQL client tools for backups:
  ```bash
  # Ubuntu/Debian
  sudo apt-get install postgresql-client
  
  # macOS
  brew install postgresql
  
  # Arch
  sudo pacman -S postgresql-libs
  ```

**"migration already in progress"**
- Another instance is running migrations
- Wait for it to complete (max 5 minutes)
- If stuck, check for hung database connections

**"migration build failed"**
- Migration file has syntax errors
- Fix the errors and try again
- Use `go build ./cmd/oslstats/migrations` to debug

#### Best Practices

1. **Always test in development first**
   - Use `make migrate-dry-run` to preview
   - Test rollback: `make migrate-rollback`
   - Verify with `make migrate-status`

2. **Write reversible migrations**
   - DOWN function should undo UP changes
   - Test rollbacks work correctly
   - Some operations (data deletion) may not be reversible

3. **Keep migrations focused**
   - One migration = one logical change
   - Don't mix schema changes with data migrations
   - Use descriptive names

4. **Production checklist**
   - ✅ Tested in development
   - ✅ Tested rollback
   - ✅ Verified backup works
   - ✅ Communicated downtime (if any)
   - ✅ Have rollback plan ready

5. **Development workflow**
   - Use `--no-backup` for speed: `make migrate-no-backup`
   - Use `--reset-db` to start fresh (loses data!)
   - Commit migrations to version control

### Configuration Management
```bash
# Generate .env template file
make genenv
# OR with custom output: make genenv OUT=.env.example

# Show environment variable documentation
make envdoc

# Show current environment values
make showenv
```

## Code Style Guidelines

### Import Organization
Organize imports in **3 groups** separated by blank lines:

```go
import (
    // 1. Standard library
    "context"
    "net/http"
    "fmt"
    
    // 2. External dependencies
    "git.haelnorr.com/h/golib/hws"
    "github.com/pkg/errors"
    "github.com/uptrace/bun"
    
    // 3. Internal packages
    "git.haelnorr.com/h/oslstats/internal/config"
    "git.haelnorr.com/h/oslstats/pkg/oauth"
)
```

### Naming Conventions

**Variables**:
- Local: `camelCase` (userAgentKey, httpServer, dbConn)
- Exported: `PascalCase` (Config, User, Token)
- Common abbreviations: `cfg`, `ctx`, `tx`, `db`, `err`, `w`, `r`

**Functions**:
- Exported: `PascalCase` (GetConfig, NewStore, GenerateState)
- Private: `camelCase` (throwError, shouldShowDetails, loadModels)
- HTTP handlers: Return `http.Handler`, use dependency injection pattern
- Database functions: Use `bun.Tx` as parameter for transactions

**Types**:
- Structs/Interfaces: `PascalCase` (Config, User, OAuthSession)
- Use `-er` suffix for interfaces (implied from usage)

**Files**:
- Prefer single word: `config.go`, `oauth.go`, `errors.go`
- Don't use snake_case except for tests: `state_test.go`
- Test files: `*_test.go` alongside source files

### Error Handling

**Always wrap errors** with context using `github.com/pkg/errors`:

```go
if err != nil {
    return errors.Wrap(err, "operation_name")
}
```

**Validate inputs at function start**:
```go
func DoSomething(cfg *Config, data string) error {
    if cfg == nil {
        return errors.New("cfg cannot be nil")
    }
    if data == "" {
        return errors.New("data cannot be empty")
    }
    // ... rest of function
}
```

**HTTP error helpers** (in handlers package):
*Note: `msg` should be a frontend presentable message*
- `throwInternalServiceError(s, w, r, msg, err)` - 500 errors
- `throwBadRequest(s, w, r, msg, err)` - 400 errors
- `throwForbidden(s, w, r, msg, err)` - 403 errors (normal)
- `throwForbiddenSecurity(s, w, r, msg, err)` - 403 security violations (WARN level)
- `throwUnauthorized(s, w, r, msg, err)` - 401 errors (normal)
- `throwUnauthorizedSecurity(s, w, r, msg, err)` - 401 security violations (WARN level)
- `throwNotFound(s, w, r, path)` - 404 errors

### Common Patterns

**HTTP Handler Pattern**:
```go
func HandlerName(server *hws.Server, deps ...) http.Handler {
    return http.HandlerFunc(
        func(w http.ResponseWriter, r *http.Request) {
            // Handler logic here
        },
    )
}
```

**Database Operation Pattern**:
```go
func GetSomething(ctx context.Context, tx bun.Tx, id int) (*Result, error) {
    result := new(Result)
    err := tx.NewSelect().
        Model(result).
        Where("id = ?", id).
        Scan(ctx)
    if err != nil {
        if err.Error() == "sql: no rows in result set" {
            return nil, nil  // Return nil, nil for not found
        }
        return nil, errors.Wrap(err, "tx.Select")
    }
    return result, nil
}
```

**Setup Function Pattern** (returns instance, cleanup func, error):
```go
func setupSomething(ctx context.Context, cfg *Config) (*Type, func() error, error) {
    instance := newInstance()
    
    err := configure(instance)
    if err != nil {
        return nil, nil, errors.Wrap(err, "configure")
    }
    
    return instance, instance.Close, nil
}
```

**Configuration Pattern** (using ezconf):
```go
type Config struct {
    Field string // ENV FIELD_NAME: Description (required/default: value)
}

func ConfigFromEnv() (any, error) {
    cfg := &Config{
        Field: env.String("FIELD_NAME", "default"),
    }
    // Validation here
    return cfg, nil
}
```

### Formatting & Types

**Formatting**:
- Use `gofmt` (standard Go formatting)

**Types**:
- Prefer explicit types over inference when it improves clarity
- Use struct tags for ORM and JSON marshaling:
  ```go
  type User struct {
      bun.BaseModel `bun:"table:users,alias:u"`
      ID           int    `bun:"id,pk,autoincrement"`
      Username     string `bun:"username,unique"`
      AccessToken  string `json:"access_token"`
  }
  ```

**Comments**:
- Document exported functions and types
- Use inline comments for ENV var documentation in Config structs
- Explain security-critical code flows

### Testing

**Test File Location**: Place `*_test.go` files alongside source files

**Test Naming**:
```go
func TestFunctionName_Scenario(t *testing.T)
func TestGenerateState_Success(t *testing.T)
func TestVerifyState_WrongUserAgentKey(t *testing.T)
```

**Test Structure**:
- Use subtests with `t.Run()` for related scenarios
- Use table-driven tests for multiple similar cases
- Create helper functions for common setup (e.g., `testConfig()`)
- Test happy paths, error cases, edge cases, and security properties

**Test Categories** (from pkg/oauth/state_test.go example):
1. Happy path tests
2. Error handling (nil params, empty fields, malformed input)
3. Security tests (MITM, CSRF, replay attacks, tampering)
4. Edge cases (concurrency, constant-time comparison)
5. Integration tests (round-trip verification)

### Security

**Critical Practices**:
- Use `crypto/subtle.ConstantTimeCompare` for cryptographic comparisons
- Implement CSRF protection via state tokens
- Store sensitive cookies as HttpOnly
- Use separate logging levels for security violations (WARN)
- Validate all inputs at function boundaries
- Use parameterized queries (Bun ORM handles this)
- Never commit secrets (.env, keys/ are gitignored)

## Project Structure

```
oslstats/
├── cmd/oslstats/          # Application entry point
│   ├── main.go            # Entry point with flag parsing
│   ├── run.go             # Server initialization & graceful shutdown
│   ├── httpserver.go      # HTTP server setup
│   ├── routes.go          # Route registration
│   ├── middleware.go      # Middleware registration
│   ├── auth.go            # Authentication setup
│   └── db.go              # Database connection & migrations
├── internal/              # Private application code
│   ├── config/            # Configuration aggregation
│   ├── db/                # Database models & queries (Bun ORM)
│   ├── discord/           # Discord OAuth integration
│   ├── handlers/          # HTTP request handlers
│   ├── session/           # Session store (in-memory)
│   └── view/              # Templ templates
│       ├── component/     # Reusable UI components
│       ├── layout/        # Page layouts
│       └── page/          # Full pages
├── pkg/                   # Reusable packages
│   ├── contexts/          # Context key definitions
│   ├── embedfs/           # Embedded static files
│   └── oauth/             # OAuth state management
├── bin/                   # Compiled binaries (gitignored)
├── keys/                  # Private keys (gitignored)
├── tmp/                   # Air hot reload temp files (gitignored)
├── Makefile               # Build automation
├── .air.toml              # Hot reload configuration
└── go.mod                 # Go module definition
```

## Key Dependencies

- **git.haelnorr.com/h/golib/*** - Custom libraries (env, ezconf, hlog, hws, hwsauth, cookies, jwt)
- **github.com/a-h/templ** - Type-safe HTML templating
- **github.com/uptrace/bun** - PostgreSQL ORM
- **github.com/bwmarrin/discordgo** - Discord API client
- **github.com/pkg/errors** - Error wrapping (use this, not fmt.Errorf)
- **github.com/joho/godotenv** - .env file loading

## Notes for AI Agents

1. **Never commit** .env files, keys/, or generated files (*_templ.go, output.css)
2. **Database operations** should use `bun.Tx` for transaction safety
3. **Templates** are written in templ, not Go html/template - run `templ generate` after changes
4. **Static files** are embedded via `//go:embed` - check pkg/embedfs/
5. **Error messages** should be descriptive and use errors.Wrap for context
6. **Security is critical** - especially in OAuth flows (see pkg/oauth/state_test.go for examples)
7. **Air proxy** runs on port 3000 during development; app runs on 3333
8. **Test coverage** is currently limited - prioritize testing security-critical code
9. **Configuration** uses ezconf pattern - see internal/*/ezconf.go files for examples
10. **Graceful shutdown** is implemented in cmd/oslstats/run.go - follow this pattern
11. When in plan mode, always use the interactive question tool if available