oslstats/AGENTS.md

AGENTS.md - Developer Guide for oslstats

This document provides guidelines for AI coding agents and developers working on the oslstats codebase.

Project Overview

Module: git.haelnorr.com/h/oslstats
Language: Go 1.25.5
Architecture: Web application with Discord OAuth, PostgreSQL database, templ templates
Key Technologies: Bun ORM, templ, TailwindCSS, custom golib libraries

Build, Test, and Development Commands

Building

NEVER BUILD MANUALLY

# Full production build (tailwind → templ → go generate → go build)
make build

# Build and run
make run

# Clean build artifacts
make clean

Development Mode

# Watch mode with hot reload (templ, air, tailwindcss in parallel)
make dev

# Development server runs on:
# - Proxy: http://localhost:3000 (use this)
# - App: http://localhost:3333 (internal)

Testing

# Run all tests
go test ./...

# Run tests for a specific package
go test ./pkg/oauth

# Run a single test function
go test ./pkg/oauth -run TestGenerateState_Success

# Run tests with verbose output
go test -v ./pkg/oauth

# Run tests with coverage
go test -cover ./...
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

Database Migrations

oslstats uses Bun's migration framework for safe, incremental schema changes.

Quick Reference

# Show migration status
make migrate-status

# Preview pending migrations (dry-run)
make migrate-dry-run

# Run pending migrations (with automatic backup)
make migrate

# Rollback last migration group
make migrate-rollback

# Create new migration
make migrate-create NAME=add_email_to_users

# Dev: Run migrations without backup (faster)
make migrate-no-backup

# Dev: Reset database (DESTRUCTIVE - deletes all data)
make reset-db

Creating a New Migration

Example: Adding an email field to users table

1. Generate migration file:

   make migrate-create NAME=add_email_to_users
   

   Creates: cmd/oslstats/migrations/20250124150030_add_email_to_users.go

2. Edit the migration file:

   package migrations
   
   import (
       "context"
       "github.com/uptrace/bun"
   )
   
   func init() {
       Migrations.MustRegister(
           // UP: Add email column
           func(ctx context.Context, db *bun.DB) error {
               _, err := db.ExecContext(ctx, 
                   "ALTER TABLE users ADD COLUMN email VARCHAR(255)")
               return err
           },
           // DOWN: Remove email column (for rollback)
           func(ctx context.Context, db *bun.DB) error {
               _, err := db.ExecContext(ctx, 
                   "ALTER TABLE users DROP COLUMN email")
               return err
           },
       )
   }
   

3. Update the model (internal/db/user.go):

   type User struct {
       bun.BaseModel `bun:"table:users,alias:u"`
   
       ID        int    `bun:"id,pk,autoincrement"`
       Username  string `bun:"username,unique"`
       Email     string `bun:"email"`          // NEW FIELD
       CreatedAt int64  `bun:"created_at"`
       DiscordID string `bun:"discord_id,unique"`
   }
   

4. Preview the migration (optional):

   make migrate-dry-run
   

   Output:

   [INFO] Pending migrations (dry-run):
     📋 20250124150030_add_email_to_users
   [INFO] Would migrate to group 2
   

5. Apply the migration:

   make migrate
   

   Output:

   [INFO] Step 1/5: Validating migrations...
   [INFO] Migration validation passed ✓
   [INFO] Step 2/5: Checking for pending migrations...
   [INFO] Step 3/5: Creating backup...
   [INFO] Backup created: backups/20250124_150145_pre_migration.sql.gz (2.3 MB)
   [INFO] Step 4/5: Acquiring migration lock...
   [INFO] Migration lock acquired
   [INFO] Step 5/5: Applying migrations...
   [INFO] Migrated to group 2
     ✅ 20250124150030_add_email_to_users
   [INFO] Migration lock released
   

Migration Patterns

Create Table:

// UP
_, err := db.NewCreateTable().
    Model((*db.Game)(nil)).
    Exec(ctx)
return err

// DOWN
_, err := db.NewDropTable().
    Model((*db.Game)(nil)).
    IfExists().
    Exec(ctx)
return err

Add Column:

// UP
_, err := db.ExecContext(ctx, 
    "ALTER TABLE users ADD COLUMN email VARCHAR(255)")
return err

// DOWN
_, err := db.ExecContext(ctx, 
    "ALTER TABLE users DROP COLUMN email")
return err

Create Index:

// UP
_, err := db.NewCreateIndex().
    Model((*db.User)(nil)).
    Index("idx_username").
    Column("username").
    Exec(ctx)
return err

// DOWN
_, err := db.ExecContext(ctx, 
    "DROP INDEX IF EXISTS idx_username")
return err

Data Migration:

// UP
_, err := db.NewUpdate().
    Model((*db.User)(nil)).
    Set("status = ?", "active").
    Where("status IS NULL").
    Exec(ctx)
return err

// DOWN (if reversible)
_, err := db.NewUpdate().
    Model((*db.User)(nil)).
    Set("status = NULL").
    Where("status = ?", "active").
    Exec(ctx)
return err

Safety Features

✅ Automatic Backups

• Compressed with gzip (saves ~80% space)
• Created before every migration and rollback
• Stored in backups/ directory
• Retention policy keeps last 10 backups
• Graceful fallback if pg_dump not installed

✅ Migration Locking

• PostgreSQL advisory locks prevent concurrent migrations
• 5-minute timeout prevents hung locks
• Safe for multi-instance deployments

✅ Validation

• Pre-migration go build check ensures migrations compile
• Dry-run mode previews changes without applying
• Status command shows exactly what's applied

✅ Rollback Support

• Every migration must have a DOWN function
• Rollbacks are grouped (undo last batch together)
• Automatic backup before rollback

Environment Variables

# Backup directory (default: backups)
DB_BACKUP_DIR=backups

# Number of backups to keep (default: 10)
DB_BACKUP_RETENTION=10

Troubleshooting

"pg_dump not found"

• Migrations will still run, but backups will be skipped
• Install PostgreSQL client tools for backups:

  # Ubuntu/Debian
  sudo apt-get install postgresql-client
  
  # macOS
  brew install postgresql
  
  # Arch
  sudo pacman -S postgresql-libs
  

"migration already in progress"

• Another instance is running migrations
• Wait for it to complete (max 5 minutes)
• If stuck, check for hung database connections

"migration build failed"

• Migration file has syntax errors
• Fix the errors and try again
• Use go build ./cmd/oslstats/migrations to debug

Best Practices

1. Always test in development first

   • Use make migrate-dry-run to preview
   • Test rollback: make migrate-rollback
   • Verify with make migrate-status

2. Write reversible migrations

   • DOWN function should undo UP changes
   • Test rollbacks work correctly
   • Some operations (data deletion) may not be reversible

3. Keep migrations focused

   • One migration = one logical change
   • Don't mix schema changes with data migrations
   • Use descriptive names

4. Production checklist

   • ✅ Tested in development
   • ✅ Tested rollback
   • ✅ Verified backup works
   • ✅ Communicated downtime (if any)
   • ✅ Have rollback plan ready

5. Development workflow

   • Use --no-backup for speed: make migrate-no-backup
   • Use --reset-db to start fresh (loses data!)
   • Commit migrations to version control

Configuration Management

# Generate .env template file
make genenv
# OR with custom output: make genenv OUT=.env.example

# Show environment variable documentation
make envdoc

# Show current environment values
make showenv

Code Style Guidelines

Import Organization

Organize imports in 3 groups separated by blank lines:

import (
    // 1. Standard library
    "context"
    "net/http"
    "fmt"
    
    // 2. External dependencies
    "git.haelnorr.com/h/golib/hws"
    "github.com/pkg/errors"
    "github.com/uptrace/bun"
    
    // 3. Internal packages
    "git.haelnorr.com/h/oslstats/internal/config"
    "git.haelnorr.com/h/oslstats/pkg/oauth"
)

Naming Conventions

Variables:

• Local: camelCase (userAgentKey, httpServer, dbConn)
• Exported: PascalCase (Config, User, Token)
• Common abbreviations: cfg, ctx, tx, db, err, w, r

Functions:

• Exported: PascalCase (GetConfig, NewStore, GenerateState)
• Private: camelCase (throwError, shouldShowDetails, loadModels)
• HTTP handlers: Return http.Handler, use dependency injection pattern
• Database functions: Use bun.Tx as parameter for transactions

Types:

• Structs/Interfaces: PascalCase (Config, User, OAuthSession)
• Use -er suffix for interfaces (implied from usage)

Files:

• Prefer single word: config.go, oauth.go, errors.go
• Don't use snake_case except for tests: state_test.go
• Test files: *_test.go alongside source files

Error Handling

Always wrap errors with context using github.com/pkg/errors:

if err != nil {
    return errors.Wrap(err, "operation_name")
}

Validate inputs at function start:

func DoSomething(cfg *Config, data string) error {
    if cfg == nil {
        return errors.New("cfg cannot be nil")
    }
    if data == "" {
        return errors.New("data cannot be empty")
    }
    // ... rest of function
}

HTTP error helpers (in handlers package): Note: msg should be a frontend presentable message

• throwInternalServiceError(s, w, r, msg, err) - 500 errors
• throwBadRequest(s, w, r, msg, err) - 400 errors
• throwForbidden(s, w, r, msg, err) - 403 errors (normal)
• throwForbiddenSecurity(s, w, r, msg, err) - 403 security violations (WARN level)
• throwUnauthorized(s, w, r, msg, err) - 401 errors (normal)
• throwUnauthorizedSecurity(s, w, r, msg, err) - 401 security violations (WARN level)
• throwNotFound(s, w, r, path) - 404 errors

Common Patterns

HTTP Handler Pattern:

func HandlerName(server *hws.Server, deps ...) http.Handler {
    return http.HandlerFunc(
        func(w http.ResponseWriter, r *http.Request) {
            // Handler logic here
        },
    )
}

Database Operation Pattern:

func GetSomething(ctx context.Context, tx bun.Tx, id int) (*Result, error) {
    result := new(Result)
    err := tx.NewSelect().
        Model(result).
        Where("id = ?", id).
        Scan(ctx)
    if err != nil {
        if err.Error() == "sql: no rows in result set" {
            return nil, nil  // Return nil, nil for not found
        }
        return nil, errors.Wrap(err, "tx.Select")
    }
    return result, nil
}

Setup Function Pattern (returns instance, cleanup func, error):

func setupSomething(ctx context.Context, cfg *Config) (*Type, func() error, error) {
    instance := newInstance()
    
    err := configure(instance)
    if err != nil {
        return nil, nil, errors.Wrap(err, "configure")
    }
    
    return instance, instance.Close, nil
}

Configuration Pattern (using ezconf):

type Config struct {
    Field string // ENV FIELD_NAME: Description (required/default: value)
}

func ConfigFromEnv() (any, error) {
    cfg := &Config{
        Field: env.String("FIELD_NAME", "default"),
    }
    // Validation here
    return cfg, nil
}

Formatting & Types

Formatting:

• Use gofmt (standard Go formatting)

Types:

• Prefer explicit types over inference when it improves clarity
• Use struct tags for ORM and JSON marshaling:

  type User struct {
      bun.BaseModel `bun:"table:users,alias:u"`
      ID           int    `bun:"id,pk,autoincrement"`
      Username     string `bun:"username,unique"`
      AccessToken  string `json:"access_token"`
  }
  

Comments:

• Document exported functions and types
• Use inline comments for ENV var documentation in Config structs
• Explain security-critical code flows

Testing

Test File Location: Place *_test.go files alongside source files

Test Naming:

func TestFunctionName_Scenario(t *testing.T)
func TestGenerateState_Success(t *testing.T)
func TestVerifyState_WrongUserAgentKey(t *testing.T)

Test Structure:

• Use subtests with t.Run() for related scenarios
• Use table-driven tests for multiple similar cases
• Create helper functions for common setup (e.g., testConfig())
• Test happy paths, error cases, edge cases, and security properties

Test Categories (from pkg/oauth/state_test.go example):

1. Happy path tests
2. Error handling (nil params, empty fields, malformed input)
3. Security tests (MITM, CSRF, replay attacks, tampering)
4. Edge cases (concurrency, constant-time comparison)
5. Integration tests (round-trip verification)

Security

Critical Practices:

• Use crypto/subtle.ConstantTimeCompare for cryptographic comparisons
• Implement CSRF protection via state tokens
• Store sensitive cookies as HttpOnly
• Use separate logging levels for security violations (WARN)
• Validate all inputs at function boundaries
• Use parameterized queries (Bun ORM handles this)
• Never commit secrets (.env, keys/ are gitignored)

Project Structure

oslstats/
├── cmd/oslstats/          # Application entry point
│   ├── main.go            # Entry point with flag parsing
│   ├── run.go             # Server initialization & graceful shutdown
│   ├── httpserver.go      # HTTP server setup
│   ├── routes.go          # Route registration
│   ├── middleware.go      # Middleware registration
│   ├── auth.go            # Authentication setup
│   └── db.go              # Database connection & migrations
├── internal/              # Private application code
│   ├── config/            # Configuration aggregation
│   ├── db/                # Database models & queries (Bun ORM)
│   ├── discord/           # Discord OAuth integration
│   ├── handlers/          # HTTP request handlers
│   ├── session/           # Session store (in-memory)
│   └── view/              # Templ templates
│       ├── component/     # Reusable UI components
│       ├── layout/        # Page layouts
│       └── page/          # Full pages
├── pkg/                   # Reusable packages
│   ├── contexts/          # Context key definitions
│   ├── embedfs/           # Embedded static files
│   └── oauth/             # OAuth state management
├── bin/                   # Compiled binaries (gitignored)
├── keys/                  # Private keys (gitignored)
├── tmp/                   # Air hot reload temp files (gitignored)
├── Makefile               # Build automation
├── .air.toml              # Hot reload configuration
└── go.mod                 # Go module definition

Key Dependencies

• git.haelnorr.com/h/golib/* - Custom libraries (env, ezconf, hlog, hws, hwsauth, cookies, jwt)
• github.com/a-h/templ - Type-safe HTML templating
• github.com/uptrace/bun - PostgreSQL ORM
• github.com/bwmarrin/discordgo - Discord API client
• github.com/pkg/errors - Error wrapping (use this, not fmt.Errorf)
• github.com/joho/godotenv - .env file loading

Notes for AI Agents

1. Never commit .env files, keys/, or generated files (*_templ.go, output.css)
2. Database operations should use bun.Tx for transaction safety
3. Templates are written in templ, not Go html/template - run templ generate after changes
4. Static files are embedded via //go:embed - check pkg/embedfs/
5. Error messages should be descriptive and use errors.Wrap for context
6. Security is critical - especially in OAuth flows (see pkg/oauth/state_test.go for examples)
7. Air proxy runs on port 3000 during development; app runs on 3333
8. Test coverage is currently limited - prioritize testing security-critical code
9. Configuration uses ezconf pattern - see internal/*/ezconf.go files for examples
10. Graceful shutdown is implemented in cmd/oslstats/run.go - follow this pattern
11. When in plan mode, always use the interactive question tool if available