vibe coded the shit out of a db migration system

This commit is contained in:
2026-01-24 16:21:28 +11:00
parent e7801afd07
commit 038f8fd1a2
15 changed files with 1649 additions and 40 deletions

264
AGENTS.md
View File

@@ -53,14 +53,270 @@ go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out
```
### Database
### Database Migrations
**oslstats uses Bun's migration framework for safe, incremental schema changes.**
#### Quick Reference
```bash
# Run migrations
# Show migration status
make migrate-status
# Preview pending migrations (dry-run)
make migrate-dry-run
# Run pending migrations (with automatic backup)
make migrate
# OR
./bin/oslstats --migrate
# Rollback last migration group
make migrate-rollback
# Create new migration
make migrate-create NAME=add_email_to_users
# Dev: Run migrations without backup (faster)
make migrate-no-backup
# Dev: Reset database (DESTRUCTIVE - deletes all data)
make reset-db
```
#### Creating a New Migration
**Example: Adding an email field to users table**
1. **Generate migration file:**
```bash
make migrate-create NAME=add_email_to_users
```
Creates: `cmd/oslstats/migrations/20250124150030_add_email_to_users.go`
2. **Edit the migration file:**
```go
package migrations
import (
"context"
"github.com/uptrace/bun"
)
func init() {
Migrations.MustRegister(
// UP: Add email column
func(ctx context.Context, db *bun.DB) error {
_, err := db.ExecContext(ctx,
"ALTER TABLE users ADD COLUMN email VARCHAR(255)")
return err
},
// DOWN: Remove email column (for rollback)
func(ctx context.Context, db *bun.DB) error {
_, err := db.ExecContext(ctx,
"ALTER TABLE users DROP COLUMN email")
return err
},
)
}
```
3. **Update the model** (`internal/db/user.go`):
```go
type User struct {
bun.BaseModel `bun:"table:users,alias:u"`
ID int `bun:"id,pk,autoincrement"`
Username string `bun:"username,unique"`
Email string `bun:"email"` // NEW FIELD
CreatedAt int64 `bun:"created_at"`
DiscordID string `bun:"discord_id,unique"`
}
```
4. **Preview the migration (optional):**
```bash
make migrate-dry-run
```
Output:
```
[INFO] Pending migrations (dry-run):
📋 20250124150030_add_email_to_users
[INFO] Would migrate to group 2
```
5. **Apply the migration:**
```bash
make migrate
```
Output:
```
[INFO] Step 1/5: Validating migrations...
[INFO] Migration validation passed ✓
[INFO] Step 2/5: Checking for pending migrations...
[INFO] Step 3/5: Creating backup...
[INFO] Backup created: backups/20250124_150145_pre_migration.sql.gz (2.3 MB)
[INFO] Step 4/5: Acquiring migration lock...
[INFO] Migration lock acquired
[INFO] Step 5/5: Applying migrations...
[INFO] Migrated to group 2
✅ 20250124150030_add_email_to_users
[INFO] Migration lock released
```
#### Migration Patterns
**Create Table:**
```go
// UP
_, err := db.NewCreateTable().
Model((*db.Game)(nil)).
Exec(ctx)
return err
// DOWN
_, err := db.NewDropTable().
Model((*db.Game)(nil)).
IfExists().
Exec(ctx)
return err
```
**Add Column:**
```go
// UP
_, err := db.ExecContext(ctx,
"ALTER TABLE users ADD COLUMN email VARCHAR(255)")
return err
// DOWN
_, err := db.ExecContext(ctx,
"ALTER TABLE users DROP COLUMN email")
return err
```
**Create Index:**
```go
// UP
_, err := db.NewCreateIndex().
Model((*db.User)(nil)).
Index("idx_username").
Column("username").
Exec(ctx)
return err
// DOWN
_, err := db.ExecContext(ctx,
"DROP INDEX IF EXISTS idx_username")
return err
```
**Data Migration:**
```go
// UP
_, err := db.NewUpdate().
Model((*db.User)(nil)).
Set("status = ?", "active").
Where("status IS NULL").
Exec(ctx)
return err
// DOWN (if reversible)
_, err := db.NewUpdate().
Model((*db.User)(nil)).
Set("status = NULL").
Where("status = ?", "active").
Exec(ctx)
return err
```
#### Safety Features
✅ **Automatic Backups**
- Compressed with gzip (saves ~80% space)
- Created before every migration and rollback
- Stored in `backups/` directory
- Retention policy keeps last 10 backups
- Graceful fallback if `pg_dump` not installed
✅ **Migration Locking**
- PostgreSQL advisory locks prevent concurrent migrations
- 5-minute timeout prevents hung locks
- Safe for multi-instance deployments
✅ **Validation**
- Pre-migration `go build` check ensures migrations compile
- Dry-run mode previews changes without applying
- Status command shows exactly what's applied
✅ **Rollback Support**
- Every migration must have a DOWN function
- Rollbacks are grouped (undo last batch together)
- Automatic backup before rollback
#### Environment Variables
```bash
# Backup directory (default: backups)
DB_BACKUP_DIR=backups
# Number of backups to keep (default: 10)
DB_BACKUP_RETENTION=10
```
#### Troubleshooting
**"pg_dump not found"**
- Migrations will still run, but backups will be skipped
- Install PostgreSQL client tools for backups:
```bash
# Ubuntu/Debian
sudo apt-get install postgresql-client
# macOS
brew install postgresql
# Arch
sudo pacman -S postgresql-libs
```
**"migration already in progress"**
- Another instance is running migrations
- Wait for it to complete (max 5 minutes)
- If stuck, check for hung database connections
**"migration build failed"**
- Migration file has syntax errors
- Fix the errors and try again
- Use `go build ./cmd/oslstats/migrations` to debug
#### Best Practices
1. **Always test in development first**
- Use `make migrate-dry-run` to preview
- Test rollback: `make migrate-rollback`
- Verify with `make migrate-status`
2. **Write reversible migrations**
- DOWN function should undo UP changes
- Test rollbacks work correctly
- Some operations (data deletion) may not be reversible
3. **Keep migrations focused**
- One migration = one logical change
- Don't mix schema changes with data migrations
- Use descriptive names
4. **Production checklist**
- ✅ Tested in development
- ✅ Tested rollback
- ✅ Verified backup works
- ✅ Communicated downtime (if any)
- ✅ Have rollback plan ready
5. **Development workflow**
- Use `--no-backup` for speed: `make migrate-no-backup`
- Use `--reset-db` to start fresh (loses data!)
- Commit migrations to version control
### Configuration Management
```bash
# Generate .env template file