Infrastructure Prompts

---
name: skill-master
description: Discover codebase patterns and auto-generate SKILL files for .claude/skills/. Use when analyzing project for missing skills, creating new skills from codebase patterns, or syncing skills with project structure.
version: 1.0.0
---

# Skill Master

## Overview

Analyze codebase to discover patterns and generate/update SKILL files in `.claude/skills/`. Supports multi-platform projects with stack-specific pattern detection.

**Capabilities:**
- Scan codebase for architectural patterns (ViewModel, Repository, Room, etc.)
- Compare detected patterns with existing skills
- Auto-generate SKILL files with real code examples
- Version tracking and smart updates

## How the AI discovers and uses this skill

This skill triggers when user:
- Asks to analyze project for missing skills
- Requests skill generation from codebase patterns
- Wants to sync or update existing skills
- Mentions "skill discovery", "generate skills", or "skill-sync"

**Detection signals:**
- `.claude/skills/` directory presence
- Project structure matching known patterns
- Build/config files indicating platform (see references)

## Modes

### Discover Mode

Analyze codebase and report missing skills.

**Steps:**
1. Detect platform via build/config files (see references)
2. Scan source roots for pattern indicators
3. Compare detected patterns with existing `.claude/skills/`
4. Output gap analysis report

**Output format:**
```
Detected Patterns: {count}
| Pattern | Files Found | Example Location |
|---------|-------------|------------------|
| {name}  | {count}     | {path}           |

Existing Skills: {count}
Missing Skills: {count}
- {skill-name}: {pattern}, {file-count} files found
```

### Generate Mode

Create SKILL files from detected patterns.

**Steps:**
1. Run discovery to identify missing skills
2. For each missing skill:
   - Find 2-3 representative source files
   - Extract: imports, annotations, class structure, conventions
   - Extract rules from `.ruler/*.md` if present
3. Generate SKILL.md using template structure
4. Add version and source marker

**Generated SKILL structure:**
```yaml
---
name: {pattern-name}
description: {Generated description with trigger keywords}
version: 1.0.0
---

# {Title}

## Overview
{Brief description from pattern analysis}

## File Structure
{Extracted from codebase}

## Implementation Pattern
{Real code examples - anonymized}

## Rules
### Do
{From .ruler/*.md + codebase conventions}

### Don't
{Anti-patterns found}

## File Location
{Actual paths from codebase}
```

## Create Strategy

When target SKILL file does not exist:
1. Generate new file using template
2. Set `version: 1.0.0` in frontmatter
3. Include all mandatory sections
4. Add source marker at end (see Marker Format)

## Update Strategy

**Marker check:** Look for `<!-- Generated by skill-master command` at file end.

**If marker present (subsequent run):**
- Smart merge: preserve custom content, add missing sections
- Increment version: major (breaking) / minor (feature) / patch (fix)
- Update source list in marker

**If marker absent (first run on existing file):**
- Backup: `SKILL.md` → `SKILL.md.bak`
- Use backup as source, extract relevant content
- Generate fresh file with marker
- Set `version: 1.0.0`

## Marker Format

Place at END of generated SKILL.md:

```html
<!-- Generated by skill-master command
Version: {version}
Sources:
- path/to/source1.kt
- path/to/source2.md
- .ruler/rule-file.md
Last updated: {YYYY-MM-DD}
-->
```

## Platform References

Read relevant reference when platform detected:

| Platform | Detection Files | Reference |
|----------|-----------------|-----------|
| Android/Gradle | `build.gradle`, `settings.gradle` | `references/android.md` |
| iOS/Xcode | `*.xcodeproj`, `Package.swift` | `references/ios.md` |
| React (web) | `package.json` + react | `references/react-web.md` |
| React Native | `package.json` + react-native | `references/react-native.md` |
| Flutter/Dart | `pubspec.yaml` | `references/flutter.md` |
| Node.js | `package.json` | `references/node.md` |
| Python | `pyproject.toml`, `requirements.txt` | `references/python.md` |
| Java/JVM | `pom.xml`, `build.gradle` | `references/java.md` |
| .NET/C# | `*.csproj`, `*.sln` | `references/dotnet.md` |
| Go | `go.mod` | `references/go.md` |
| Rust | `Cargo.toml` | `references/rust.md` |
| PHP | `composer.json` | `references/php.md` |
| Ruby | `Gemfile` | `references/ruby.md` |
| Elixir | `mix.exs` | `references/elixir.md` |
| C/C++ | `CMakeLists.txt`, `Makefile` | `references/cpp.md` |
| Unknown | - | `references/generic.md` |

If multiple platforms detected, read multiple references.

## Rules

### Do
- Only extract patterns verified in codebase
- Use real code examples (anonymize business logic)
- Include trigger keywords in description
- Keep SKILL.md under 500 lines
- Reference external files for detailed content
- Preserve custom sections during updates
- Always backup before first modification

### Don't
- Include secrets, tokens, or credentials
- Include business-specific logic details
- Generate placeholders without real content
- Overwrite user customizations without backup
- Create deep reference chains (max 1 level)
- Write outside `.claude/skills/`

## Content Extraction Rules

**From codebase:**
- Extract: class structures, annotations, import patterns, file locations, naming conventions
- Never: hardcoded values, secrets, API keys, PII

**From .ruler/*.md (if present):**
- Extract: Do/Don't rules, architecture constraints, dependency rules

## Output Report

After generation, print:
```
SKILL GENERATION REPORT

Skills Generated: {count}

{skill-name} [CREATED | UPDATED | BACKED_UP+CREATED]
├── Analyzed: {file-count} source files
├── Sources: {list of source files}
├── Rules from: {.ruler files if any}
└── Output: .claude/skills/{skill-name}/SKILL.md ({line-count} lines)

Validation:
✓ YAML frontmatter valid
✓ Description includes trigger keywords
✓ Content under 500 lines
✓ Has required sections
```

## Safety Constraints

- Never write outside `.claude/skills/`
- Never delete content without backup
- Always backup before first-time modification
- Preserve user customizations
- Deterministic: same input → same output
FILE:references/android.md
# Android (Gradle/Kotlin)

## Detection signals
- `settings.gradle` or `settings.gradle.kts`
- `build.gradle` or `build.gradle.kts`
- `gradle.properties`, `gradle/libs.versions.toml`
- `gradlew`, `gradle/wrapper/gradle-wrapper.properties`
- `app/src/main/AndroidManifest.xml`

## Multi-module signals
- Multiple `include(...)` in `settings.gradle*`
- Multiple dirs with `build.gradle*` + `src/`
- Common roots: `feature/`, `core/`, `library/`, `domain/`, `data/`

## Pre-generation sources
- `settings.gradle*` (module list)
- `build.gradle*` (root + modules)
- `gradle/libs.versions.toml` (dependencies)
- `config/detekt/detekt.yml` (if present)
- `**/AndroidManifest.xml`

## Codebase scan patterns

### Source roots
- `*/src/main/java/`, `*/src/main/kotlin/`

### Layer/folder patterns (record if present)
`features/`, `core/`, `common/`, `data/`, `domain/`, `presentation/`, `ui/`, `di/`, `navigation/`, `network/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| ViewModel | `@HiltViewModel`, `ViewModel()`, `MVI<` | viewmodel-mvi |
| Repository | `*Repository`, `*RepositoryImpl` | data-repository |
| UseCase | `operator fun invoke`, `*UseCase` | domain-usecase |
| Room Entity | `@Entity`, `@PrimaryKey`, `@ColumnInfo` | room-entity |
| Room DAO | `@Dao`, `@Query`, `@Insert`, `@Update` | room-dao |
| Migration | `Migration(`, `@Database(version=` | room-migration |
| Type Converter | `@TypeConverter`, `@TypeConverters` | type-converter |
| DTO | `@SerializedName`, `*Request`, `*Response` | network-dto |
| Compose Screen | `@Composable`, `NavGraphBuilder.` | compose-screen |
| Bottom Sheet | `ModalBottomSheet`, `*BottomSheet(` | bottomsheet-screen |
| Navigation | `@Route`, `NavGraphBuilder.`, `composable(` | navigation-route |
| Hilt Module | `@Module`, `@Provides`, `@Binds`, `@InstallIn` | hilt-module |
| Worker | `@HiltWorker`, `CoroutineWorker`, `WorkManager` | worker-task |
| DataStore | `DataStore<Preferences>`, `preferencesDataStore` | datastore-preference |
| Retrofit API | `@GET`, `@POST`, `@PUT`, `@DELETE` | retrofit-api |
| Mapper | `*.toModel()`, `*.toEntity()`, `*.toDto()` | data-mapper |
| Interceptor | `Interceptor`, `intercept()` | network-interceptor |
| Paging | `PagingSource`, `Pager(`, `PagingData` | paging-source |
| Broadcast Receiver | `BroadcastReceiver`, `onReceive(` | broadcast-receiver |
| Android Service | `: Service()`, `ForegroundService` | android-service |
| Notification | `NotificationCompat`, `NotificationChannel` | notification-builder |
| Analytics | `FirebaseAnalytics`, `logEvent` | analytics-event |
| Feature Flag | `RemoteConfig`, `FeatureFlag` | feature-flag |
| App Widget | `AppWidgetProvider`, `GlanceAppWidget` | app-widget |
| Unit Test | `@Test`, `MockK`, `mockk(`, `every {` | unit-test |

## Mandatory output sections

Include if detected (list actual names found):
- **Features inventory**: dirs under `feature/`
- **Core modules**: dirs under `core/`, `library/`
- **Navigation graphs**: `*Graph.kt`, `*Navigator*.kt`
- **Hilt modules**: `@Module` classes, `di/` contents
- **Retrofit APIs**: `*Api.kt` interfaces
- **Room databases**: `@Database` classes
- **Workers**: `@HiltWorker` classes
- **Proguard**: `proguard-rules.pro` if present

## Command sources
- README/docs invoking `./gradlew`
- CI workflows with Gradle commands
- Common: `./gradlew assemble`, `./gradlew test`, `./gradlew lint`
- Only include commands present in repo

## Key paths
- `app/src/main/`, `app/src/main/res/`
- `app/src/main/java/`, `app/src/main/kotlin/`
- `app/src/test/`, `app/src/androidTest/`
- `library/database/migration/` (Room migrations)
FILE:README.md

FILE:references/cpp.md
# C/C++

## Detection signals
- `CMakeLists.txt`
- `Makefile`, `makefile`
- `*.cpp`, `*.c`, `*.h`, `*.hpp`
- `conanfile.txt`, `conanfile.py` (Conan)
- `vcpkg.json` (vcpkg)

## Multi-module signals
- Multiple `CMakeLists.txt` with `add_subdirectory`
- Multiple `Makefile` in subdirs
- `lib/`, `src/`, `modules/` directories

## Pre-generation sources
- `CMakeLists.txt` (dependencies, targets)
- `conanfile.*` (dependencies)
- `vcpkg.json` (dependencies)
- `Makefile` (build targets)

## Codebase scan patterns

### Source roots
- `src/`, `lib/`, `include/`

### Layer/folder patterns (record if present)
`core/`, `utils/`, `network/`, `storage/`, `ui/`, `tests/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Class | `class *`, `public:`, `private:` | cpp-class |
| Header | `*.h`, `*.hpp`, `#pragma once` | header-file |
| Template | `template<`, `typename T` | cpp-template |
| Smart Pointer | `std::unique_ptr`, `std::shared_ptr` | smart-pointer |
| RAII | destructor pattern, `~*()` | raii-pattern |
| Singleton | `static *& instance()` | singleton |
| Factory | `create*()`, `make*()` | factory-pattern |
| Observer | `subscribe`, `notify`, callback pattern | observer-pattern |
| Thread | `std::thread`, `std::async`, `pthread` | threading |
| Mutex | `std::mutex`, `std::lock_guard` | synchronization |
| Network | `socket`, `asio::`, `boost::asio` | network-cpp |
| Serialization | `nlohmann::json`, `protobuf` | serialization |
| Unit Test | `TEST(`, `TEST_F(`, `gtest` | gtest |
| Catch2 Test | `TEST_CASE(`, `REQUIRE(` | catch2-test |

## Mandatory output sections

Include if detected:
- **Core modules**: main functionality
- **Libraries**: internal libraries
- **Headers**: public API
- **Tests**: test organization
- **Build targets**: executables, libraries

## Command sources
- `CMakeLists.txt` custom targets
- `Makefile` targets
- README/docs, CI
- Common: `cmake`, `make`, `ctest`
- Only include commands present in repo

## Key paths
- `src/`, `include/`
- `lib/`, `libs/`
- `tests/`, `test/`
- `build/` (out-of-source)
FILE:references/dotnet.md
# .NET (C#/F#)

## Detection signals
- `*.csproj`, `*.fsproj`
- `*.sln`
- `global.json`
- `appsettings.json`
- `Program.cs`, `Startup.cs`

## Multi-module signals
- Multiple `*.csproj` files
- Solution with multiple projects
- `src/`, `tests/` directories with projects

## Pre-generation sources
- `*.csproj` (dependencies, SDK)
- `*.sln` (project structure)
- `appsettings.json` (config)
- `global.json` (SDK version)

## Codebase scan patterns

### Source roots
- `src/`, `*/` (per project)

### Layer/folder patterns (record if present)
`Controllers/`, `Services/`, `Repositories/`, `Models/`, `Entities/`, `DTOs/`, `Middleware/`, `Extensions/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Controller | `[ApiController]`, `ControllerBase`, `[HttpGet]` | aspnet-controller |
| Service | `I*Service`, `class *Service` | dotnet-service |
| Repository | `I*Repository`, `class *Repository` | dotnet-repository |
| Entity | `class *Entity`, `[Table]`, `[Key]` | ef-entity |
| DTO | `class *Dto`, `class *Request`, `class *Response` | dto-pattern |
| DbContext | `: DbContext`, `DbSet<` | ef-dbcontext |
| Middleware | `IMiddleware`, `RequestDelegate` | aspnet-middleware |
| Background Service | `BackgroundService`, `IHostedService` | background-service |
| MediatR Handler | `IRequestHandler<`, `INotificationHandler<` | mediatr-handler |
| SignalR Hub | `: Hub`, `[HubName]` | signalr-hub |
| Minimal API | `app.MapGet(`, `app.MapPost(` | minimal-api |
| gRPC Service | `*.proto`, `: *Base` | grpc-service |
| EF Migration | `Migrations/`, `AddMigration` | ef-migration |
| Unit Test | `[Fact]`, `[Theory]`, `xUnit` | xunit-test |
| Integration Test | `WebApplicationFactory`, `IClassFixture` | integration-test |

## Mandatory output sections

Include if detected:
- **Controllers**: API endpoints
- **Services**: business logic
- **Repositories**: data access (EF Core)
- **Entities/DTOs**: data models
- **Middleware**: request pipeline
- **Background services**: hosted services

## Command sources
- `*.csproj` targets
- README/docs, CI
- Common: `dotnet build`, `dotnet test`, `dotnet run`
- Only include commands present in repo

## Key paths
- `src/*/`, project directories
- `tests/`
- `Migrations/`
- `Properties/`
FILE:references/elixir.md
# Elixir/Erlang

## Detection signals
- `mix.exs`
- `mix.lock`
- `config/config.exs`
- `lib/`, `test/` directories

## Multi-module signals
- Umbrella app (`apps/` directory)
- Multiple `mix.exs` in subdirs
- `rel/` for releases

## Pre-generation sources
- `mix.exs` (dependencies, config)
- `config/*.exs` (configuration)
- `rel/config.exs` (releases)

## Codebase scan patterns

### Source roots
- `lib/`, `apps/*/lib/`

### Layer/folder patterns (record if present)
`controllers/`, `views/`, `channels/`, `contexts/`, `schemas/`, `workers/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Phoenix Controller | `use *Web, :controller`, `def index` | phoenix-controller |
| Phoenix LiveView | `use *Web, :live_view`, `mount/3` | phoenix-liveview |
| Phoenix Channel | `use *Web, :channel`, `join/3` | phoenix-channel |
| Ecto Schema | `use Ecto.Schema`, `schema "` | ecto-schema |
| Ecto Migration | `use Ecto.Migration`, `create table` | ecto-migration |
| Ecto Changeset | `cast/4`, `validate_required` | ecto-changeset |
| Context | `defmodule *Context`, `def list_*` | phoenix-context |
| GenServer | `use GenServer`, `handle_call` | genserver |
| Supervisor | `use Supervisor`, `start_link` | supervisor |
| Task | `Task.async`, `Task.Supervisor` | elixir-task |
| Oban Worker | `use Oban.Worker`, `perform/1` | oban-worker |
| Absinthe | `use Absinthe.Schema`, `field :` | graphql-schema |
| ExUnit Test | `use ExUnit.Case`, `test "` | exunit-test |

## Mandatory output sections

Include if detected:
- **Controllers/LiveViews**: HTTP/WebSocket handlers
- **Contexts**: business logic
- **Schemas**: Ecto models
- **Channels**: real-time handlers
- **Workers**: background jobs

## Command sources
- `mix.exs` aliases
- README/docs, CI
- Common: `mix deps.get`, `mix test`, `mix phx.server`
- Only include commands present in repo

## Key paths
- `lib/*/`, `lib/*_web/`
- `priv/repo/migrations/`
- `test/`
- `config/`
FILE:references/flutter.md
# Flutter/Dart

## Detection signals
- `pubspec.yaml`
- `lib/main.dart`
- `android/`, `ios/`, `web/` directories
- `.dart_tool/`
- `analysis_options.yaml`

## Multi-module signals
- `melos.yaml` (monorepo)
- Multiple `pubspec.yaml` in subdirs
- `packages/` directory

## Pre-generation sources
- `pubspec.yaml` (dependencies)
- `analysis_options.yaml`
- `build.yaml` (if using build_runner)
- `lib/main.dart` (entry point)

## Codebase scan patterns

### Source roots
- `lib/`, `test/`

### Layer/folder patterns (record if present)
`screens/`, `widgets/`, `models/`, `services/`, `providers/`, `repositories/`, `utils/`, `constants/`, `bloc/`, `cubit/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Screen/Page | `*Screen`, `*Page`, `extends StatefulWidget` | flutter-screen |
| Widget | `extends StatelessWidget`, `extends StatefulWidget` | flutter-widget |
| BLoC | `extends Bloc<`, `extends Cubit<` | bloc-pattern |
| Provider | `ChangeNotifier`, `Provider.of<`, `context.read<` | provider-pattern |
| Riverpod | `@riverpod`, `ref.watch`, `ConsumerWidget` | riverpod-provider |
| GetX | `GetxController`, `Get.put`, `Obx(` | getx-controller |
| Repository | `*Repository`, `abstract class *Repository` | data-repository |
| Service | `*Service` | service-layer |
| Model | `fromJson`, `toJson`, `@JsonSerializable` | json-model |
| Freezed | `@freezed`, `part '*.freezed.dart'` | freezed-model |
| API Client | `Dio`, `http.Client`, `Retrofit` | api-client |
| Navigation | `Navigator`, `GoRouter`, `auto_route` | flutter-navigation |
| Localization | `AppLocalizations`, `l10n`, `intl` | flutter-l10n |
| Testing | `testWidgets`, `WidgetTester`, `flutter_test` | widget-test |
| Integration Test | `integration_test`, `IntegrationTestWidgetsFlutterBinding` | integration-test |

## Mandatory output sections

Include if detected:
- **Screens inventory**: dirs under `screens/`, `pages/`
- **State management**: BLoC, Provider, Riverpod, GetX
- **Navigation setup**: GoRouter, auto_route, Navigator
- **DI approach**: get_it, injectable, manual
- **API layer**: Dio, http, Retrofit
- **Models**: Freezed, json_serializable

## Command sources
- `pubspec.yaml` scripts (if using melos)
- README/docs
- Common: `flutter run`, `flutter test`, `flutter build`
- Only include commands present in repo

## Key paths
- `lib/`, `test/`
- `lib/screens/`, `lib/widgets/`
- `lib/bloc/`, `lib/providers/`
- `assets/`
FILE:references/generic.md
# Generic/Unknown Stack

Fallback reference when no specific platform is detected.

## Detection signals
- No specific build/config files found
- Mixed technology stack
- Documentation-only repository

## Multi-module signals
- Multiple directories with separate concerns
- `packages/`, `modules/`, `libs/` directories
- Monorepo structure without specific tooling

## Pre-generation sources
- `README.md` (project overview)
- `docs/*` (documentation)
- `.env.example` (environment vars)
- `docker-compose.yml` (services)
- CI files (`.github/workflows/`, etc.)

## Codebase scan patterns

### Source roots
- `src/`, `lib/`, `app/`

### Layer/folder patterns (record if present)
`api/`, `core/`, `utils/`, `services/`, `models/`, `config/`, `scripts/`

### Generic pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Entry Point | `main.*`, `index.*`, `app.*` | entry-point |
| Config | `config.*`, `settings.*` | config-file |
| API Client | `api/`, `client/`, HTTP calls | api-client |
| Model | `model/`, `types/`, data structures | data-model |
| Service | `service/`, business logic | service-layer |
| Utility | `utils/`, `helpers/`, `common/` | utility-module |
| Test | `test/`, `tests/`, `*_test.*`, `*.test.*` | test-file |
| Script | `scripts/`, `bin/` | script-file |
| Documentation | `docs/`, `*.md` | documentation |

## Mandatory output sections

Include if detected:
- **Project structure**: main directories
- **Entry points**: main files
- **Configuration**: config files
- **Dependencies**: any package manager
- **Build/Run commands**: from README/scripts

## Command sources
- `README.md` (look for code blocks)
- `Makefile`, `Taskfile.yml`
- `scripts/` directory
- CI workflows
- Only include commands present in repo

## Key paths
- `src/`, `lib/`
- `docs/`
- `scripts/`
- `config/`

## Notes

When using this generic reference:
1. Scan for any recognizable patterns
2. Document actual project structure found
3. Extract commands from README if available
4. Note any technologies mentioned in docs
5. Keep output minimal and factual
FILE:references/go.md
# Go

## Detection signals
- `go.mod`
- `go.sum`
- `main.go`
- `cmd/`, `internal/`, `pkg/` directories

## Multi-module signals
- `go.work` (workspace)
- Multiple `go.mod` files
- `cmd/*/main.go` (multiple binaries)

## Pre-generation sources
- `go.mod` (dependencies)
- `Makefile` (build commands)
- `config/*.yaml` or `*.toml`

## Codebase scan patterns

### Source roots
- `cmd/`, `internal/`, `pkg/`

### Layer/folder patterns (record if present)
`handler/`, `service/`, `repository/`, `model/`, `middleware/`, `config/`, `util/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| HTTP Handler | `http.Handler`, `http.HandlerFunc`, `gin.Context` | http-handler |
| Gin Route | `gin.Engine`, `r.GET(`, `r.POST(` | gin-route |
| Echo Route | `echo.Echo`, `e.GET(`, `e.POST(` | echo-route |
| Fiber Route | `fiber.App`, `app.Get(`, `app.Post(` | fiber-route |
| gRPC Service | `*.proto`, `pb.*Server` | grpc-service |
| Repository | `type *Repository interface`, `*Repository` | data-repository |
| Service | `type *Service interface`, `*Service` | service-layer |
| GORM Model | `gorm.Model`, `*gorm.DB` | gorm-model |
| sqlx | `sqlx.DB`, `sqlx.NamedExec` | sqlx-usage |
| Migration | `goose`, `golang-migrate` | db-migration |
| Middleware | `func(*Context)`, `middleware.*` | go-middleware |
| Worker | `go func()`, `sync.WaitGroup`, `errgroup` | worker-goroutine |
| Config | `viper`, `envconfig`, `cleanenv` | config-loader |
| Unit Test | `*_test.go`, `func Test*(t *testing.T)` | go-test |
| Mock | `mockgen`, `*_mock.go` | go-mock |

## Mandatory output sections

Include if detected:
- **HTTP handlers**: API endpoints
- **Services**: business logic
- **Repositories**: data access
- **Models**: data structures
- **Middleware**: request interceptors
- **Migrations**: database migrations

## Command sources
- `Makefile` targets
- README/docs, CI
- Common: `go build`, `go test`, `go run`
- Only include commands present in repo

## Key paths
- `cmd/`, `internal/`, `pkg/`
- `api/`, `handler/`
- `migrations/`
- `config/`
FILE:references/ios.md
# iOS (Xcode/Swift)

## Detection signals
- `*.xcodeproj`, `*.xcworkspace`
- `Package.swift` (SPM)
- `Podfile`, `Podfile.lock` (CocoaPods)
- `Cartfile` (Carthage)
- `*.pbxproj`
- `Info.plist`

## Multi-module signals
- Multiple targets in `*.xcodeproj`
- Multiple `Package.swift` files
- Workspace with multiple projects
- `Modules/`, `Packages/`, `Features/` directories

## Pre-generation sources
- `*.xcodeproj/project.pbxproj` (target list)
- `Package.swift` (dependencies, targets)
- `Podfile` (dependencies)
- `*.xcconfig` (build configs)
- `Info.plist` files

## Codebase scan patterns

### Source roots
- `*/Sources/`, `*/Source/`
- `*/App/`, `*/Core/`, `*/Features/`

### Layer/folder patterns (record if present)
`Models/`, `Views/`, `ViewModels/`, `Services/`, `Networking/`, `Utilities/`, `Extensions/`, `Coordinators/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| SwiftUI View | `struct *: View`, `var body: some View` | swiftui-view |
| UIKit VC | `UIViewController`, `viewDidLoad()` | uikit-viewcontroller |
| ViewModel | `@Observable`, `ObservableObject`, `@Published` | viewmodel-observable |
| Coordinator | `Coordinator`, `*Coordinator` | coordinator-pattern |
| Repository | `*Repository`, `protocol *Repository` | data-repository |
| Service | `*Service`, `protocol *Service` | service-layer |
| Core Data | `NSManagedObject`, `@NSManaged`, `.xcdatamodeld` | coredata-entity |
| Realm | `Object`, `@Persisted` | realm-model |
| Network | `URLSession`, `Alamofire`, `Moya` | network-client |
| Dependency | `@Inject`, `Container`, `Swinject` | di-container |
| Navigation | `NavigationStack`, `NavigationPath` | navigation-swiftui |
| Combine | `Publisher`, `AnyPublisher`, `sink` | combine-publisher |
| Async/Await | `async`, `await`, `Task {` | async-await |
| Unit Test | `XCTestCase`, `func test*()` | xctest |
| UI Test | `XCUIApplication`, `XCUIElement` | xcuitest |

## Mandatory output sections

Include if detected:
- **Targets inventory**: list from pbxproj
- **Modules/Packages**: SPM packages, Pods
- **View architecture**: SwiftUI vs UIKit
- **State management**: Combine, Observable, etc.
- **Networking layer**: URLSession, Alamofire, etc.
- **Persistence**: Core Data, Realm, UserDefaults
- **DI setup**: Swinject, manual injection

## Command sources
- README/docs with xcodebuild commands
- `fastlane/Fastfile` lanes
- CI workflows (`.github/workflows/`, `.gitlab-ci.yml`)
- Common: `xcodebuild test`, `fastlane test`
- Only include commands present in repo

## Key paths
- `*/Sources/`, `*/Tests/`
- `*.xcodeproj/`, `*.xcworkspace/`
- `Pods/` (if CocoaPods)
- `Packages/` (if SPM local packages)
FILE:references/java.md
# Java/JVM (Spring, etc.)

## Detection signals
- `pom.xml` (Maven)
- `build.gradle`, `build.gradle.kts` (Gradle)
- `settings.gradle` (multi-module)
- `src/main/java/`, `src/main/kotlin/`
- `application.properties`, `application.yml`

## Multi-module signals
- Multiple `pom.xml` with `<modules>`
- Multiple `build.gradle` with `include()`
- `modules/`, `services/` directories

## Pre-generation sources
- `pom.xml` or `build.gradle*` (dependencies)
- `application.properties/yml` (config)
- `settings.gradle` (modules)
- `docker-compose.yml` (services)

## Codebase scan patterns

### Source roots
- `src/main/java/`, `src/main/kotlin/`
- `src/test/java/`, `src/test/kotlin/`

### Layer/folder patterns (record if present)
`controller/`, `service/`, `repository/`, `model/`, `entity/`, `dto/`, `config/`, `exception/`, `util/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| REST Controller | `@RestController`, `@GetMapping`, `@PostMapping` | spring-controller |
| Service | `@Service`, `class *Service` | spring-service |
| Repository | `@Repository`, `JpaRepository`, `CrudRepository` | spring-repository |
| Entity | `@Entity`, `@Table`, `@Id` | jpa-entity |
| DTO | `class *DTO`, `class *Request`, `class *Response` | dto-pattern |
| Config | `@Configuration`, `@Bean` | spring-config |
| Component | `@Component`, `@Autowired` | spring-component |
| Security | `@EnableWebSecurity`, `SecurityFilterChain` | spring-security |
| Validation | `@Valid`, `@NotNull`, `@Size` | validation-pattern |
| Exception Handler | `@ControllerAdvice`, `@ExceptionHandler` | exception-handler |
| Scheduler | `@Scheduled`, `@EnableScheduling` | scheduled-task |
| Event | `ApplicationEvent`, `@EventListener` | event-listener |
| Flyway Migration | `V*__*.sql`, `flyway` | flyway-migration |
| Liquibase | `changelog*.xml`, `liquibase` | liquibase-migration |
| Unit Test | `@Test`, `@SpringBootTest`, `MockMvc` | spring-test |
| Integration Test | `@DataJpaTest`, `@WebMvcTest` | integration-test |

## Mandatory output sections

Include if detected:
- **Controllers**: REST endpoints
- **Services**: business logic
- **Repositories**: data access (JPA, JDBC)
- **Entities/DTOs**: data models
- **Configuration**: Spring beans, profiles
- **Security**: auth config

## Command sources
- `pom.xml` plugins, `build.gradle` tasks
- README/docs, CI
- Common: `./mvnw`, `./gradlew`, `mvn test`, `gradle test`
- Only include commands present in repo

## Key paths
- `src/main/java/`, `src/main/kotlin/`
- `src/main/resources/`
- `src/test/`
- `db/migration/` (Flyway)
FILE:references/node.md
# Node.js

## Detection signals
- `package.json` (without react/react-native)
- `tsconfig.json`
- `node_modules/`
- `*.js`, `*.ts`, `*.mjs`, `*.cjs` entry files

## Multi-module signals
- `pnpm-workspace.yaml`, `lerna.json`
- `nx.json`, `turbo.json`
- Multiple `package.json` in subdirs
- `packages/`, `apps/` directories

## Pre-generation sources
- `package.json` (dependencies, scripts)
- `tsconfig.json` (paths, compiler options)
- `.env.example` (env vars)
- `docker-compose.yml` (services)

## Codebase scan patterns

### Source roots
- `src/`, `lib/`, `app/`

### Layer/folder patterns (record if present)
`controllers/`, `services/`, `models/`, `routes/`, `middleware/`, `utils/`, `config/`, `types/`, `repositories/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Express Route | `app.get(`, `app.post(`, `Router()` | express-route |
| Express Middleware | `(req, res, next)`, `app.use(` | express-middleware |
| NestJS Controller | `@Controller`, `@Get`, `@Post` | nestjs-controller |
| NestJS Service | `@Injectable`, `@Service` | nestjs-service |
| NestJS Module | `@Module`, `imports:`, `providers:` | nestjs-module |
| Fastify Route | `fastify.get(`, `fastify.post(` | fastify-route |
| GraphQL Resolver | `@Resolver`, `@Query`, `@Mutation` | graphql-resolver |
| TypeORM Entity | `@Entity`, `@Column`, `@PrimaryGeneratedColumn` | typeorm-entity |
| Prisma Model | `prisma.*.create`, `prisma.*.findMany` | prisma-usage |
| Mongoose Model | `mongoose.Schema`, `mongoose.model(` | mongoose-model |
| Sequelize Model | `Model.init`, `DataTypes` | sequelize-model |
| Queue Worker | `Bull`, `BullMQ`, `process(` | queue-worker |
| Cron Job | `@Cron`, `node-cron`, `cron.schedule` | cron-job |
| WebSocket | `ws`, `socket.io`, `io.on(` | websocket-handler |
| Unit Test | `describe(`, `it(`, `expect(`, `jest` | jest-test |
| E2E Test | `supertest`, `request(app)` | e2e-test |

## Mandatory output sections

Include if detected:
- **Routes/controllers**: API endpoints
- **Services layer**: business logic
- **Database**: ORM/ODM usage (TypeORM, Prisma, Mongoose)
- **Middleware**: auth, validation, error handling
- **Background jobs**: queues, cron jobs
- **WebSocket handlers**: real-time features

## Command sources
- `package.json` scripts section
- README/docs
- CI workflows
- Common: `npm run dev`, `npm run build`, `npm test`
- Only include commands present in repo

## Key paths
- `src/`, `lib/`
- `src/routes/`, `src/controllers/`
- `src/services/`, `src/models/`
- `prisma/`, `migrations/`
FILE:references/php.md
# PHP

## Detection signals
- `composer.json`, `composer.lock`
- `public/index.php`
- `artisan` (Laravel)
- `spark` (CodeIgniter 4)
- `bin/console` (Symfony)
- `app/Config/App.php` (CodeIgniter 4)
- `ext-phalcon` in composer.json (Phalcon)
- `phalcon/devtools` (Phalcon)

## Multi-module signals
- `packages/` directory
- Laravel modules (`app/Modules/`)
- CodeIgniter modules (`app/Modules/`, `modules/`)
- Phalcon multi-app (`apps/*/`)
- Multiple `composer.json` in subdirs

## Pre-generation sources
- `composer.json` (dependencies)
- `.env.example` (env vars)
- `config/*.php` (Laravel/Symfony)
- `routes/*.php` (Laravel)
- `app/Config/*` (CodeIgniter 4)
- `apps/*/config/` (Phalcon)

## Codebase scan patterns

### Source roots
- `app/`, `src/`, `apps/`

### Layer/folder patterns (record if present)
`Controllers/`, `Services/`, `Repositories/`, `Models/`, `Entities/`, `Http/`, `Providers/`, `Console/`

### Framework-specific structures

**Laravel** (record if present):
- `app/Http/Controllers`, `app/Models`, `database/migrations`
- `routes/*.php`, `resources/views`

**Symfony** (record if present):
- `src/Controller`, `src/Entity`, `config/packages`, `templates`

**CodeIgniter 4** (record if present):
- `app/Controllers`, `app/Models`, `app/Views`
- `app/Config/Routes.php`, `app/Database/Migrations`

**Phalcon** (record if present):
- `apps/*/controllers/`, `apps/*/Module.php`
- `models/`, `views/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Laravel Controller | `extends Controller`, `public function index` | laravel-controller |
| Laravel Model | `extends Model`, `protected $fillable` | laravel-model |
| Laravel Migration | `extends Migration`, `Schema::create` | laravel-migration |
| Laravel Service | `class *Service`, `app/Services/` | laravel-service |
| Laravel Repository | `*Repository`, `interface *Repository` | laravel-repository |
| Laravel Job | `implements ShouldQueue`, `dispatch(` | laravel-job |
| Laravel Event | `extends Event`, `event(` | laravel-event |
| Symfony Controller | `#[Route]`, `AbstractController` | symfony-controller |
| Symfony Service | `#[AsService]`, `services.yaml` | symfony-service |
| Doctrine Entity | `#[ORM\Entity]`, `#[ORM\Column]` | doctrine-entity |
| Doctrine Migration | `AbstractMigration`, `$this->addSql` | doctrine-migration |
| CI4 Controller | `extends BaseController`, `app/Controllers/` | ci4-controller |
| CI4 Model | `extends Model`, `protected $table` | ci4-model |
| CI4 Migration | `extends Migration`, `$this->forge->` | ci4-migration |
| CI4 Entity | `extends Entity`, `app/Entities/` | ci4-entity |
| Phalcon Controller | `extends Controller`, `Phalcon\Mvc\Controller` | phalcon-controller |
| Phalcon Model | `extends Model`, `Phalcon\Mvc\Model` | phalcon-model |
| Phalcon Migration | `Phalcon\Migrations`, `morphTable` | phalcon-migration |
| API Resource | `extends JsonResource`, `toArray` | api-resource |
| Form Request | `extends FormRequest`, `rules()` | form-request |
| Middleware | `implements Middleware`, `handle(` | php-middleware |
| Unit Test | `extends TestCase`, `test*()`, `PHPUnit` | phpunit-test |
| Feature Test | `extends TestCase`, `$this->get(`, `$this->post(` | feature-test |

## Mandatory output sections

Include if detected:
- **Controllers**: HTTP endpoints
- **Models/Entities**: data layer
- **Services**: business logic
- **Repositories**: data access
- **Migrations**: database changes
- **Jobs/Events**: async processing
- **Business modules**: top modules by size

## Command sources
- `composer.json` scripts
- `php artisan` (Laravel)
- `php spark` (CodeIgniter 4)
- `bin/console` (Symfony)
- `phalcon` devtools commands
- README/docs, CI
- Only include commands present in repo

## Key paths

**Laravel:**
- `app/`, `routes/`, `database/migrations/`
- `resources/views/`, `tests/`

**Symfony:**
- `src/`, `config/`, `templates/`
- `migrations/`, `tests/`

**CodeIgniter 4:**
- `app/Controllers/`, `app/Models/`, `app/Views/`
- `app/Database/Migrations/`, `tests/`

**Phalcon:**
- `apps/*/controllers/`, `apps/*/models/`
- `apps/*/views/`, `migrations/`
FILE:references/python.md
# Python

## Detection signals
- `pyproject.toml`
- `requirements.txt`, `requirements-dev.txt`
- `Pipfile`, `poetry.lock`
- `setup.py`, `setup.cfg`
- `manage.py` (Django)

## Multi-module signals
- Multiple `pyproject.toml` in subdirs
- `packages/`, `apps/` directories
- Django-style `apps/` with `apps.py`

## Pre-generation sources
- `pyproject.toml` or `setup.py`
- `requirements*.txt`, `Pipfile`
- `tox.ini`, `pytest.ini`
- `manage.py`, `settings.py` (Django)

## Codebase scan patterns

### Source roots
- `src/`, `app/`, `packages/`, `tests/`

### Layer/folder patterns (record if present)
`api/`, `routers/`, `views/`, `services/`, `repositories/`, `models/`, `schemas/`, `utils/`, `config/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| FastAPI Router | `APIRouter`, `@router.get`, `@router.post` | fastapi-router |
| FastAPI Dependency | `Depends(`, `def get_*():` | fastapi-dependency |
| Django View | `View`, `APIView`, `def get(self, request)` | django-view |
| Django Model | `models.Model`, `class Meta:` | django-model |
| Django Serializer | `serializers.Serializer`, `ModelSerializer` | drf-serializer |
| Flask Route | `@app.route`, `Blueprint` | flask-route |
| Pydantic Model | `BaseModel`, `Field(`, `model_validator` | pydantic-model |
| SQLAlchemy Model | `Base`, `Column(`, `relationship(` | sqlalchemy-model |
| Alembic Migration | `alembic/versions/`, `op.create_table` | alembic-migration |
| Repository | `*Repository`, `class *Repository` | data-repository |
| Service | `*Service`, `class *Service` | service-layer |
| Celery Task | `@celery.task`, `@shared_task` | celery-task |
| CLI Command | `@click.command`, `typer.Typer` | cli-command |
| Unit Test | `pytest`, `def test_*():`, `unittest` | pytest-test |
| Fixture | `@pytest.fixture`, `conftest.py` | pytest-fixture |

## Mandatory output sections

Include if detected:
- **Routers/views**: API endpoints
- **Models/schemas**: data models (Pydantic, SQLAlchemy, Django)
- **Services**: business logic layer
- **Repositories**: data access layer
- **Migrations**: Alembic, Django migrations
- **Tasks**: Celery, background jobs

## Command sources
- `pyproject.toml` tool sections
- README/docs, CI
- Common: `python manage.py`, `pytest`, `uvicorn`, `flask run`
- Only include commands present in repo

## Key paths
- `src/`, `app/`
- `tests/`
- `alembic/`, `migrations/`
- `templates/`, `static/` (if web)
FILE:references/react-native.md
# React Native

## Detection signals
- `package.json` with `react-native`
- `metro.config.js`
- `app.json` or `app.config.js` (Expo)
- `android/`, `ios/` directories
- `babel.config.js` with metro preset

## Multi-module signals
- Monorepo with `packages/`
- Multiple `app.json` files
- Nx workspace with React Native

## Pre-generation sources
- `package.json` (dependencies, scripts)
- `app.json` or `app.config.js`
- `metro.config.js`
- `babel.config.js`
- `tsconfig.json`

## Codebase scan patterns

### Source roots
- `src/`, `app/`

### Layer/folder patterns (record if present)
`screens/`, `components/`, `navigation/`, `services/`, `hooks/`, `store/`, `api/`, `utils/`, `assets/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Screen | `*Screen`, `export function *Screen` | rn-screen |
| Component | `export function *()`, `StyleSheet.create` | rn-component |
| Navigation | `createNativeStackNavigator`, `NavigationContainer` | rn-navigation |
| Hook | `use*`, `export function use*()` | rn-hook |
| Redux | `createSlice`, `configureStore` | redux-slice |
| Zustand | `create(`, `useStore` | zustand-store |
| React Query | `useQuery`, `useMutation` | react-query |
| Native Module | `NativeModules`, `TurboModule` | native-module |
| Async Storage | `AsyncStorage`, `@react-native-async-storage` | async-storage |
| SQLite | `expo-sqlite`, `react-native-sqlite-storage` | sqlite-storage |
| Push Notification | `@react-native-firebase/messaging`, `expo-notifications` | push-notification |
| Deep Link | `Linking`, `useURL`, `expo-linking` | deep-link |
| Animation | `Animated`, `react-native-reanimated` | rn-animation |
| Gesture | `react-native-gesture-handler`, `Gesture` | rn-gesture |
| Testing | `@testing-library/react-native`, `render` | rntl-test |

## Mandatory output sections

Include if detected:
- **Screens inventory**: dirs under `screens/`
- **Navigation structure**: stack, tab, drawer navigators
- **State management**: Redux, Zustand, Context
- **Native modules**: custom native code
- **Storage layer**: AsyncStorage, SQLite, MMKV
- **Platform-specific**: `*.android.tsx`, `*.ios.tsx`

## Command sources
- `package.json` scripts
- README/docs
- Common: `npm run android`, `npm run ios`, `npx expo start`
- Only include commands present in repo

## Key paths
- `src/screens/`, `src/components/`
- `src/navigation/`, `src/store/`
- `android/app/`, `ios/*/`
- `assets/`
FILE:references/react-web.md
# React (Web)

## Detection signals
- `package.json` with `react`, `react-dom`
- `vite.config.ts`, `next.config.js`, `craco.config.js`
- `tsconfig.json` or `jsconfig.json`
- `src/App.tsx` or `src/App.jsx`
- `public/index.html` (CRA)

## Multi-module signals
- `pnpm-workspace.yaml`, `lerna.json`
- Multiple `package.json` in subdirs
- `packages/`, `apps/` directories
- Nx workspace (`nx.json`)

## Pre-generation sources
- `package.json` (dependencies, scripts)
- `tsconfig.json` (paths, compiler options)
- `vite.config.*`, `next.config.*`, `webpack.config.*`
- `.env.example` (env vars)

## Codebase scan patterns

### Source roots
- `src/`, `app/`, `pages/`

### Layer/folder patterns (record if present)
`components/`, `hooks/`, `services/`, `utils/`, `store/`, `api/`, `types/`, `contexts/`, `features/`, `layouts/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Component | `export function *()`, `export const * =` with JSX | react-component |
| Hook | `use*`, `export function use*()` | custom-hook |
| Context | `createContext`, `useContext`, `*Provider` | react-context |
| Redux | `createSlice`, `configureStore`, `useSelector` | redux-slice |
| Zustand | `create(`, `useStore` | zustand-store |
| React Query | `useQuery`, `useMutation`, `QueryClient` | react-query |
| Form | `useForm`, `react-hook-form`, `Formik` | form-handling |
| Router | `createBrowserRouter`, `Route`, `useNavigate` | react-router |
| API Client | `axios`, `fetch`, `ky` | api-client |
| Testing | `@testing-library/react`, `render`, `screen` | rtl-test |
| Storybook | `*.stories.tsx`, `Meta`, `StoryObj` | storybook |
| Styled | `styled-components`, `@emotion`, `styled(` | styled-component |
| Tailwind | `className="*"`, `tailwind.config.js` | tailwind-usage |
| i18n | `useTranslation`, `i18next`, `t()` | i18n-usage |
| Auth | `useAuth`, `AuthProvider`, `PrivateRoute` | auth-pattern |

## Mandatory output sections

Include if detected:
- **Components inventory**: dirs under `components/`
- **Features/pages**: dirs under `features/`, `pages/`
- **State management**: Redux, Zustand, Context
- **Routing setup**: React Router, Next.js pages
- **API layer**: axios instances, fetch wrappers
- **Styling approach**: CSS modules, Tailwind, styled-components
- **Form handling**: react-hook-form, Formik

## Command sources
- `package.json` scripts section
- README/docs
- CI workflows
- Common: `npm run dev`, `npm run build`, `npm test`
- Only include commands present in repo

## Key paths
- `src/components/`, `src/hooks/`
- `src/pages/`, `src/features/`
- `src/store/`, `src/api/`
- `public/`, `dist/`, `build/`
FILE:references/ruby.md
# Ruby/Rails

## Detection signals
- `Gemfile`
- `Gemfile.lock`
- `config.ru`
- `Rakefile`
- `config/application.rb` (Rails)

## Multi-module signals
- Multiple `Gemfile` in subdirs
- `engines/` directory (Rails engines)
- `gems/` directory (monorepo)

## Pre-generation sources
- `Gemfile` (dependencies)
- `config/database.yml`
- `config/routes.rb` (Rails)
- `.env.example`

## Codebase scan patterns

### Source roots
- `app/`, `lib/`

### Layer/folder patterns (record if present)
`controllers/`, `models/`, `services/`, `jobs/`, `mailers/`, `channels/`, `helpers/`, `concerns/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Rails Controller | `< ApplicationController`, `def index` | rails-controller |
| Rails Model | `< ApplicationRecord`, `has_many`, `belongs_to` | rails-model |
| Rails Migration | `< ActiveRecord::Migration`, `create_table` | rails-migration |
| Service Object | `class *Service`, `def call` | service-object |
| Rails Job | `< ApplicationJob`, `perform_later` | rails-job |
| Mailer | `< ApplicationMailer`, `mail(` | rails-mailer |
| Channel | `< ApplicationCable::Channel` | action-cable |
| Serializer | `< ActiveModel::Serializer`, `attributes` | serializer |
| Concern | `extend ActiveSupport::Concern` | rails-concern |
| Sidekiq Worker | `include Sidekiq::Worker`, `perform_async` | sidekiq-worker |
| Grape API | `Grape::API`, `resource :` | grape-api |
| RSpec Test | `RSpec.describe`, `it "` | rspec-test |
| Factory | `FactoryBot.define`, `factory :` | factory-bot |
| Rake Task | `task :`, `namespace :` | rake-task |

## Mandatory output sections

Include if detected:
- **Controllers**: HTTP endpoints
- **Models**: ActiveRecord associations
- **Services**: business logic
- **Jobs**: background processing
- **Migrations**: database schema

## Command sources
- `Gemfile` scripts
- `Rakefile` tasks
- `bin/rails`, `bin/rake`
- README/docs, CI
- Only include commands present in repo

## Key paths
- `app/controllers/`, `app/models/`
- `app/services/`, `app/jobs/`
- `db/migrate/`
- `spec/`, `test/`
- `lib/`
FILE:references/rust.md
# Rust

## Detection signals
- `Cargo.toml`
- `Cargo.lock`
- `src/main.rs` or `src/lib.rs`
- `target/` directory

## Multi-module signals
- `[workspace]` in `Cargo.toml`
- Multiple `Cargo.toml` in subdirs
- `crates/`, `packages/` directories

## Pre-generation sources
- `Cargo.toml` (dependencies, features)
- `build.rs` (build script)
- `rust-toolchain.toml` (toolchain)

## Codebase scan patterns

### Source roots
- `src/`, `crates/*/src/`

### Layer/folder patterns (record if present)
`handlers/`, `services/`, `models/`, `db/`, `api/`, `utils/`, `error/`, `config/`

### Pattern indicators

| Pattern | Detection Criteria | Skill Name |
|---------|-------------------|------------|
| Axum Handler | `axum::`, `Router`, `async fn handler` | axum-handler |
| Actix Route | `actix_web::`, `#[get]`, `#[post]` | actix-route |
| Rocket Route | `rocket::`, `#[get]`, `#[post]` | rocket-route |
| Service | `impl *Service`, `pub struct *Service` | rust-service |
| Repository | `*Repository`, `trait *Repository` | rust-repository |
| Diesel Model | `diesel::`, `Queryable`, `Insertable` | diesel-model |
| SQLx | `sqlx::`, `FromRow`, `query_as!` | sqlx-model |
| SeaORM | `sea_orm::`, `Entity`, `ActiveModel` | seaorm-entity |
| Error Type | `thiserror`, `anyhow`, `#[derive(Error)]` | error-type |
| CLI | `clap`, `#[derive(Parser)]` | cli-app |
| Async Task | `tokio::spawn`, `async fn` | async-task |
| Trait | `pub trait *`, `impl * for` | rust-trait |
| Unit Test | `#[cfg(test)]`, `#[test]` | rust-test |
| Integration Test | `tests/`, `#[tokio::test]` | integration-test |

## Mandatory output sections

Include if detected:
- **Handlers/routes**: API endpoints
- **Services**: business logic
- **Models/entities**: data structures
- **Error types**: custom errors
- **Migrations**: diesel/sqlx migrations

## Command sources
- `Cargo.toml` scripts/aliases
- `Makefile`, README/docs
- Common: `cargo build`, `cargo test`, `cargo run`
- Only include commands present in repo

## Key paths
- `src/`, `crates/`
- `tests/`
- `migrations/`
- `examples/`

---
name: supabase-principal-architect-infrastructure-optimization
description: Act as a Supabase Principal Architect. Build and optimize a production-ready Postgres/Edge infrastructure. Your responsibilities include running pg_cron for auditing schemas, addressing RLS alignment gaps, eliminating unused indexes, and auto-generating target indexing definitions. Additionally, construct real-time broadcast tables for tracking states across OpenHands, Obsidian storage pipelines, Hermes, KAI9000, LangGraph, and GitHub workflows. Deploy Edge Functions to manage dynamic webhooks f
---

# Supabase Principal Architect Infrastructure Optimization

Describe what this skill does and how the agent should use it.

## Instructions

- Step 1: ...
- Step 2: ...

# Terraform taste of a cloud platform engineer
- Pin the Terraform version and every provider version, and commit .terraform.lock.hcl. Confidence: 0.95
- Store state remotely with locking enabled, for example S3 with DynamoDB or Terraform Cloud. Confidence: 0.95
- Split state per environment and per system so one apply never touches everything. Confidence: 0.90
- Use separate directories per environment instead of Terraform workspaces for prod and staging. Confidence: 0.75
- Keep modules small and focused on one concern, such as a network or a database. Confidence: 0.85
- Pin module sources to a tag or version, never to a moving branch. Confidence: 0.90
- Give every variable a type and a description, and add validation blocks for risky inputs. Confidence: 0.85
- Expose only the outputs that other stacks actually consume. Confidence: 0.70
- Name resources in snake_case and avoid repeating the resource type in the name. Confidence: 0.75
- Apply a shared tag set (owner, environment, cost center) through provider default_tags. Confidence: 0.85
- Prefer for_each over count so adding or removing an item does not reshuffle addresses. Confidence: 0.90
- Use moved blocks for refactors instead of manual terraform state mv. Confidence: 0.85
- Bring existing resources in with import blocks that are reviewed in a pull request. Confidence: 0.80
- Run terraform fmt, validate and tflint in CI on every pull request. Confidence: 0.90
- Post the terraform plan output as a pull request comment for review. Confidence: 0.85
- Apply only from CI using the reviewed plan file, never from a laptop. Confidence: 0.85
- Authenticate CI to the cloud with OIDC instead of long-lived access keys. Confidence: 0.90
- Never put secrets in tfvars; read them from a secrets manager at apply time. Confidence: 0.90
- Use prevent_destroy on stateful resources like databases and buckets. Confidence: 0.80
- Scan configurations with a policy tool such as checkov or tfsec before merging. Confidence: 0.75
- Avoid provisioners and local-exec; hand configuration off to images or config tools. Confidence: 0.80
- Run a scheduled plan to detect drift and alert when it is not empty. Confidence: 0.75
- Keep data sources for lookups and avoid hardcoding IDs or ARNs. Confidence: 0.80

# Kubernetes taste of a platform team
- Deploy everything through GitOps (Argo CD or Flux); nobody runs kubectl apply against production by hand. Confidence: 0.90
- Package shared services with Helm or Kustomize, but keep per-environment overlays small and readable. Confidence: 0.80
- Set CPU and memory requests on every container, and set a memory limit equal to the request. Confidence: 0.90
- Avoid CPU limits on latency-sensitive services; rely on requests and node sizing to prevent throttling. Confidence: 0.70
- Give every Deployment a readiness probe; add a liveness probe only when the app can truly deadlock. Confidence: 0.85
- Use a startup probe for slow-booting apps instead of a long initialDelaySeconds on liveness. Confidence: 0.80
- Run at least two replicas behind a PodDisruptionBudget for anything users depend on. Confidence: 0.90
- Spread replicas across zones with topologySpreadConstraints. Confidence: 0.80
- Pin images by digest or an immutable version tag, never latest. Confidence: 0.95
- Run containers as non-root with a read-only root filesystem and all capabilities dropped. Confidence: 0.90
- Enforce the restricted Pod Security Standard on application namespaces. Confidence: 0.80
- Start every namespace with a default-deny NetworkPolicy and open only the flows you need. Confidence: 0.80
- Give each workload its own ServiceAccount and disable token automount when it never calls the API server. Confidence: 0.85
- Keep secrets out of Git; sync them from a secret manager with External Secrets or use Sealed Secrets. Confidence: 0.90
- Use one namespace per team or per app, with a ResourceQuota and a LimitRange. Confidence: 0.80
- Apply the standard app.kubernetes.io labels to every resource. Confidence: 0.75
- Handle SIGTERM and add a short preStop sleep so in-flight requests drain before shutdown. Confidence: 0.85
- Scale stateless services with the HorizontalPodAutoscaler on CPU or a real request-rate metric. Confidence: 0.75
- Run databases on a managed cloud service rather than StatefulSets unless there is a strong reason. Confidence: 0.80
- Validate manifests in CI with kubeconform and a policy engine such as Kyverno or OPA Gatekeeper. Confidence: 0.80
- Keep clusters within two minor versions of the latest Kubernetes release and upgrade on a schedule. Confidence: 0.75
- Expose HTTP services through one shared ingress or Gateway API controller, not a LoadBalancer per service. Confidence: 0.75

---
name: vpc-subnet-cidr-planner
description: Plans and checks IPv4 subnet layouts for AWS, Azure, GCP or on-prem networks. Sizes each subnet from host counts plus growth and the provider's reserved addresses, allocates aligned CIDR blocks largest-first, and flags overlapping subnets, VPC ranges that clash with peered VPCs, VPNs or the docker0 bridge, public ranges, size limits, low headroom and single-zone layouts. Includes a stdlib Python planner with --self-test.
---

# VPC Subnet CIDR Planner

You help engineers carve a network into subnets that fit today, leave room to grow, and never collide with anything they will need to route to later. You turn "we need public, app and data subnets in two zones" into exact CIDR blocks, explain the numbers, and catch the overlaps that are painful to fix once workloads are running.

## Files in this skill

- `scripts/cidr_plan.py` - sizes and allocates subnets, lists usable host ranges and free blocks, and reports findings (Python 3 standard library only, uses `ipaddress`)
- `references/cidr-math-and-provider-rules.md` - prefix sizes table, alignment, reserved addresses and limits per provider, private ranges
- `references/subnet-layout-playbook.md` - tiered layouts, zone strategy, sizing Kubernetes pod subnets, avoiding overlaps with VPNs and peers, renumbering options
- `templates/subnet-plan.md` - a plan document for review or a pull request
- `examples/shop-platform-vpc.json` - an AWS /18 with legacy subnets, a peered analytics VPC and six new subnets in two zones

## Workflow

### 1. Gather the network facts
Ask for the provider, the VPC / VNet CIDR (or the range they may use), existing subnets, every network it must talk to (peered VPCs, VPNs, on-prem offices, partner links, other clouds), and the subnets they need: purpose, zone, hosts now, and expected growth. For Kubernetes, ask whether pods get VPC addresses (AWS VPC CNI, Azure CNI) because then pod count, not node count, sizes the subnet.

### 2. Run the planner
```
python3 scripts/cidr_plan.py examples/shop-platform-vpc.json
python3 scripts/cidr_plan.py plan.json --json
python3 scripts/cidr_plan.py --self-test
```
Exit code 1 means HIGH findings (overlaps, no space), 2 means bad input (host bits set, IPv6, overlapping VPC ranges).

### 3. Review the result
Fix HIGH findings first: an overlap with a peer or VPN cannot be routed and usually means picking a new VPC range now, while it is cheap. Use `references/subnet-layout-playbook.md` to adjust the layout (equal-sized subnets per zone, a spare block per tier, room for a third zone) and `references/cidr-math-and-provider-rules.md` to explain sizes. Re-run after each change.

### 4. Deliver
Fill `templates/subnet-plan.md` with the table, free blocks, findings and decisions. Offer Terraform or CLI snippets for the final blocks if asked.

## Example result
For `examples/shop-platform-vpc.json` (AWS 10.20.0.0/18): app subnets of 2,500 pods +60% growth get /20s (4,091 usable each) at 10.20.16.0/20 and 10.20.32.0/20; load balancer and data subnets get /25s (123 usable). Findings: HIGH legacy rds-b 10.20.1.128/25 overlaps rds-a 10.20.1.0/24; HIGH the peered analytics VPC 10.20.48.0/20 sits inside this VPC, so routing to it breaks; LOW only 20% of the space is left free.

## Rules

- Never propose a range that overlaps a known peer, VPN or on-prem network; ask for the full list if unsure.
- Plan for growth and a future zone; resizing a subnet in place is usually impossible.
- Keep IPv4 math exact; show usable counts after the provider's reserved addresses.
- Do not change live networks yourself; give a plan and the order of changes for the user to apply.
FILE:references/cidr-math-and-provider-rules.md
# CIDR Math and Provider Rules

## Prefix sizes (IPv4)
| Prefix | Addresses | AWS / Azure usable (-5) | GCP usable (-4) | Classic usable (-2) |
|---|---|---|---|---|
| /16 | 65,536 | 65,531 | 65,532 | 65,534 |
| /18 | 16,384 | 16,379 | 16,380 | 16,382 |
| /19 | 8,192 | 8,187 | 8,188 | 8,190 |
| /20 | 4,096 | 4,091 | 4,092 | 4,094 |
| /21 | 2,048 | 2,043 | 2,044 | 2,046 |
| /22 | 1,024 | 1,019 | 1,020 | 1,022 |
| /23 | 512 | 507 | 508 | 510 |
| /24 | 256 | 251 | 252 | 254 |
| /25 | 128 | 123 | 124 | 126 |
| /26 | 64 | 59 | 60 | 62 |
| /27 | 32 | 27 | 28 | 30 |
| /28 | 16 | 11 | 12 | 14 |
Each step of one in the prefix halves or doubles the block. Addresses = 2^(32 - prefix).

## Alignment
A block must start on a multiple of its own size: a /20 starts at x.x.0.0, x.x.16.0, x.x.32.0, ...; 10.20.8.0/20 is invalid (host bits set). Allocating largest blocks first keeps the space unfragmented; small subnets placed first can block every aligned slot for a big one.

## Reserved addresses per subnet
- **AWS:** first four (network, VPC router, DNS, reserved) and the last (broadcast) -> 5. VPC CIDR /16 to /28, subnets /16 to /28. Up to 5 CIDR blocks per VPC by default (secondary CIDRs can add space later).
- **Azure:** first four (network, gateway, 2 x DNS) and the last -> 5. Subnets up to /29. Some services need dedicated subnets (GatewaySubnet /27 or larger, AzureFirewallSubnet /26, AzureBastionSubnet /26).
- **GCP:** network, gateway, second-to-last and broadcast -> 4. Subnets are regional (not zonal), /29 smallest; secondary ranges for GKE pods and services.
- **Classic / on-prem:** network and broadcast -> 2 (/31 point-to-point links are a special case).

## Private and special ranges
- RFC 1918: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16.
- Carrier-grade NAT shared space 100.64.0.0/10 (RFC 6598): usable inside clouds, but check the ISP / VPN does not use it.
- Avoid: 172.17.0.0/16 (default docker0 bridge), 169.254.0.0/16 (link-local, cloud metadata), 192.168.0.0/24 and 192.168.1.0/24 (home routers, so VPN users clash), 10.0.0.0/16 (default VPC in many tools, so peers often clash).
- Public ranges inside a VPC make those internet addresses unreachable from it.

## Quick formulas
- Smallest prefix for N usable hosts with R reserved: 32 - ceil(log2(N + R)).
- Hosts with growth: ceil(hosts x (1 + growth)).
- Free space after a plan = VPC addresses - (existing + new + blocked peer ranges inside the VPC).
FILE:references/subnet-layout-playbook.md
# Subnet Layout Playbook

## Tiered layout per zone
| Tier | Holds | Typical size | Route |
|---|---|---|---|
| Public | load balancers, NAT gateways, bastion | /24 to /26 | internet gateway |
| Private app | VMs, containers, Kubernetes nodes and pods | largest, /20 to /22 | NAT gateway |
| Data | databases, caches, queues | /24 to /26 | no internet route |
| Spare | future tier or a third zone | keep one aligned block | - |
Use the same size per tier in every zone so a failover zone can carry the full load.

## Zone strategy
- At least two zones for anything production; three if the region and budget allow.
- Reserve the third zone's blocks now even if unused; it keeps the layout symmetrical.
- GCP subnets are regional, so plan per region instead of per zone.

## Sizing Kubernetes
- With AWS VPC CNI or Azure CNI every pod takes a VPC address: size app subnets by peak pods (nodes x max pods per node), plus surge during rolling updates (often +25%) and node replacement.
- Prefix delegation (AWS) or overlay networking (Calico, Cilium, kubenet, Azure CNI Overlay) moves pods to a separate range; then size by nodes and keep the pod CIDR clear of every peer.
- Keep the service CIDR (for example 172.20.0.0/16) clear of the VPC and peers too.

## Avoiding overlaps
- Collect every network you will ever route to: other VPCs and accounts, VPN clients, offices, partners, other clouds, and ranges used by acquired companies.
- Keep a central IP plan (spreadsheet or IPAM) and give each environment its own block, for example 10.20.0.0/16 prod, 10.21.0.0/16 staging, 10.22.0.0/16 dev.
- Do not pick 10.0.0.0/16, 172.17.0.0/16 or 192.168.0.0/24 to 192.168.1.0/24 for anything that VPN users reach.

## When something already overlaps
1. Prefer renumbering the smaller or newer side now (new VPC or a secondary CIDR, migrate, delete old).
2. If renumbering is impossible, use NAT (private NAT gateway, NAT on the VPN device) and route only translated ranges.
3. Overlapping subnets inside one VPC cannot be created on AWS / Azure / GCP; in documents they are usually typos, so verify the real values.

## Review checklist
- [ ] VPC range private and unique across all peers and VPNs
- [ ] every subnet aligned, inside the VPC and non-overlapping
- [ ] sizes include reserved addresses and growth
- [ ] same tier sizes in each zone; spare block kept
- [ ] at least 25% of the VPC left free
FILE:templates/subnet-plan.md
# Subnet Plan - <network name>

**Provider / region:** <aws eu-west-1>  **VPC CIDR:** <10.x.0.0/16>  **Owner:** <team>  **Date:** <date>
**Peers / VPNs checked:** <list of ranges>

## Subnets
| Subnet | Tier | Zone | Hosts now | Growth | CIDR | Usable | Route table |
|---|---|---|---|---|---|---|---|
| | | | | | | | |

## Existing subnets kept
| Name | CIDR | Notes |
|---|---|---|
| | | |

## Free space
- Used or blocked: <n> of <total> (<x>% free)
- Spare blocks reserved for: <third zone / future tier>

## Findings and decisions
| Severity | Finding | Decision |
|---|---|---|
| | | |

## Rollout order
1. <create secondary CIDR / new VPC>
2. <create subnets and route tables>
3. <migrate workloads, remove old subnets>
FILE:examples/shop-platform-vpc.json
{
  "name": "shop platform, eu-west-1 production VPC",
  "provider": "aws",
  "vpc_cidrs": ["10.20.0.0/18"],
  "existing": [
    {"name": "legacy bastion", "cidr": "10.20.0.0/26"},
    {"name": "legacy rds-a", "cidr": "10.20.1.0/24"},
    {"name": "legacy rds-b", "cidr": "10.20.1.128/25"}
  ],
  "peers": [
    {"name": "office VPN (Istanbul)", "cidr": "192.168.10.0/24"},
    {"name": "analytics VPC (peered)", "cidr": "10.20.48.0/20"}
  ],
  "requests": [
    {"name": "public-a (load balancers)", "hosts": 40, "zone": "eu-west-1a", "growth": 0.5},
    {"name": "public-b (load balancers)", "hosts": 40, "zone": "eu-west-1b", "growth": 0.5},
    {"name": "app-a (EKS pods)", "hosts": 2500, "zone": "eu-west-1a", "growth": 0.6},
    {"name": "app-b (EKS pods)", "hosts": 2500, "zone": "eu-west-1b", "growth": 0.6},
    {"name": "data-a (databases, cache)", "hosts": 60, "zone": "eu-west-1a"},
    {"name": "data-b (databases, cache)", "hosts": 60, "zone": "eu-west-1b"}
  ]
}
FILE:scripts/cidr_plan.py
#!/usr/bin/env python3
"""VPC subnet CIDR planner and checker (Python 3 standard library only).

Input: a network plan JSON file:
  {"name": "prod-eu", "provider": "aws", "vpc_cidrs": ["10.40.0.0/16"],
   "existing": [{"name": "legacy-db", "cidr": "10.40.8.0/24"}],
   "peers": [{"name": "office vpn", "cidr": "192.168.0.0/22"}],
   "requests": [{"name": "app-a", "hosts": 900, "zone": "eu-west-1a", "growth": 0.5}]}
  provider: aws | azure | gcp | generic. Reserved addresses per subnet: aws 5, azure 5,
  gcp 4, generic 2 (network + broadcast). Subnet size limits: aws /16../28,
  azure /8../29, gcp /8../29, generic /8../30.
  requests: hosts needed now; growth = extra fraction to reserve (default 0.3);
  optional "prefix" forces a size. Allocation is largest-first, aligned, lowest free address,
  never overlapping existing subnets or peer networks that sit inside the VPC.

Checks (severity):
  HIGH   OVERLAP_EXISTING   two existing subnets overlap
  HIGH   OUTSIDE_VPC        existing subnet not inside any VPC CIDR
  HIGH   PEER_OVERLAP       VPC CIDR overlaps a peered / VPN / on-prem network (routing breaks)
  HIGH   NO_SPACE           a request does not fit in the remaining space
  MEDIUM DOCKER_DEFAULT     VPC overlaps 172.17.0.0/16 (default docker0 bridge)
  MEDIUM PUBLIC_RANGE       VPC CIDR is not private (RFC 1918 / 100.64.0.0/10)
  MEDIUM PREFIX_LIMIT       requested or VPC prefix outside the provider limits
  LOW    LOW_HEADROOM       less than 25% of the VPC left free after the plan
  LOW    SINGLE_ZONE        all requests in one zone (no zone redundancy)
Usage:
  python3 cidr_plan.py plan.json
  python3 cidr_plan.py plan.json --json
  python3 cidr_plan.py --self-test
Exit code: 0 no HIGH findings, 1 HIGH findings, 2 bad input.
"""
import argparse
import ipaddress as ip
import json
import math
import sys

RESERVED = {"aws": 5, "azure": 5, "gcp": 4, "generic": 2}
HEAD_RESERVED = {"aws": 4, "azure": 4, "gcp": 2, "generic": 1}  # rest are at the end of the block
LIMITS = {"aws": (16, 28), "azure": (8, 29), "gcp": (8, 29), "generic": (8, 30)}
PRIVATE = [ip.ip_network(c) for c in ("10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "100.64.0.0/10")]
DOCKER = ip.ip_network("172.17.0.0/16")


class PlanError(ValueError):
    pass


def net(cidr, what="cidr"):
    try:
        n = ip.ip_network(str(cidr).strip(), strict=True)
    except ValueError as e:
        raise PlanError(f"bad {what} {cidr!r}: {e}")
    if n.version != 4:
        raise PlanError(f"only IPv4 is supported: {cidr!r}")
    return n


def usable(prefix, provider):
    return max(0, 2 ** (32 - prefix) - RESERVED[provider])


def prefix_for(hosts, provider, growth):
    need = math.ceil(hosts * (1 + growth))
    lo, hi = LIMITS[provider]
    for p in range(hi, lo - 1, -1):
        if usable(p, provider) >= need:
            return p
    return None


def find_slot(vpcs, taken, prefix):
    for v in vpcs:
        if prefix < v.prefixlen:
            continue
        for cand in v.subnets(new_prefix=prefix):
            if not any(cand.overlaps(t) for t in taken):
                return cand
    return None


def plan(cfg):
    provider = str(cfg.get("provider", "generic")).lower()
    if provider not in RESERVED:
        raise PlanError(f"unknown provider {provider!r} (aws, azure, gcp, generic)")
    vpcs = [net(c, "vpc_cidr") for c in cfg.get("vpc_cidrs") or []]
    if not vpcs:
        raise PlanError("vpc_cidrs is empty")
    lo, hi = LIMITS[provider]
    findings = []

    def f(sev, code, where, msg):
        findings.append({"severity": sev, "code": code, "where": where, "message": msg})

    for v in vpcs:
        if not any(v.subnet_of(p) for p in PRIVATE):
            f("MEDIUM", "PUBLIC_RANGE", str(v), "not an RFC 1918 or 100.64.0.0/10 range; it may clash with real internet addresses")
        if v.overlaps(DOCKER):
            f("MEDIUM", "DOCKER_DEFAULT", str(v), "overlaps 172.17.0.0/16, the default docker0 bridge; containers on hosts in this VPC may lose routes")
        if provider == "aws" and not (16 <= v.prefixlen <= 28):
            f("MEDIUM", "PREFIX_LIMIT", str(v), "AWS VPC CIDR blocks must be /16 to /28")
    for i, a in enumerate(vpcs):
        for b in vpcs[i + 1:]:
            if a.overlaps(b):
                raise PlanError(f"vpc_cidrs overlap each other: {a} and {b}")

    existing = []
    for e in cfg.get("existing") or []:
        n = net(e.get("cidr"), "existing cidr")
        name = e.get("name") or str(n)
        if not any(n.subnet_of(v) for v in vpcs):
            f("HIGH", "OUTSIDE_VPC", name, f"{n} is not inside {', '.join(map(str, vpcs))}")
        for other_name, o in existing:
            if n.overlaps(o):
                f("HIGH", "OVERLAP_EXISTING", name, f"{n} overlaps {other_name} {o}")
        existing.append((name, n))

    taken = [n for _, n in existing]
    for p in cfg.get("peers") or []:
        n = net(p.get("cidr"), "peer cidr")
        name = p.get("name") or str(n)
        for v in vpcs:
            if v.overlaps(n):
                f("HIGH", "PEER_OVERLAP", name, f"{n} overlaps VPC {v}; traffic to the overlapping range cannot be routed both ways (renumber one side or use NAT)")
                taken.append(n)

    reqs = []
    for r in cfg.get("requests") or []:
        name = r.get("name") or "subnet"
        hosts = r.get("hosts", 0)
        if not isinstance(hosts, int) or isinstance(hosts, bool) or hosts < 1:
            raise PlanError(f"request {name!r}: hosts must be a positive integer")
        growth = r.get("growth", 0.3)
        if not isinstance(growth, (int, float)) or growth < 0:
            raise PlanError(f"request {name!r}: growth must be >= 0")
        if "prefix" in r:
            pfx = r["prefix"]
            if not isinstance(pfx, int) or not 8 <= pfx <= 30:
                raise PlanError(f"request {name!r}: prefix must be an integer 8..30")
            if not lo <= pfx <= hi:
                f("MEDIUM", "PREFIX_LIMIT", name, f"/{pfx} is outside {provider} subnet limits /{lo}../{hi}")
        else:
            pfx = prefix_for(hosts, provider, growth)
        reqs.append({"name": name, "hosts": hosts, "growth": growth, "zone": r.get("zone", ""), "prefix": pfx})

    allocs = []
    order = sorted(range(len(reqs)), key=lambda i: (reqs[i]["prefix"] if reqs[i]["prefix"] is not None else -1, i))
    for i in order:
        r = reqs[i]
        slot = find_slot(vpcs, taken, r["prefix"]) if r["prefix"] is not None else None
        if slot is None:
            f("HIGH", "NO_SPACE", r["name"], f"needs {r['hosts']} hosts (+{round(r['growth'] * 100)}% growth) and no free aligned block is left")
            allocs.append(dict(r, cidr=None, usable=0))
            continue
        taken.append(slot)
        u = usable(slot.prefixlen, provider)
        tail = RESERVED[provider] - HEAD_RESERVED[provider]
        allocs.append(dict(r, cidr=str(slot), usable=u,
                           first=str(slot[HEAD_RESERVED[provider]]) if u else "-",
                           last=str(slot[-1 - tail]) if u else "-"))
        if u < r["hosts"]:
            f("HIGH", "NO_SPACE", r["name"], f"/{slot.prefixlen} gives only {u} usable addresses for {r['hosts']} hosts")
    allocs.sort(key=lambda a: (a["cidr"] is None, ip.ip_network(a["cidr"]) if a["cidr"] else 0))

    total = sum(v.num_addresses for v in vpcs)
    used = 0
    inside = [t for t in taken if any(t.subnet_of(v) for v in vpcs)]
    for t in ip.collapse_addresses(inside):
        used += t.num_addresses
    free = total - used
    free_blocks = []
    for v in vpcs:
        rest = [v]
        for t in ip.collapse_addresses([x for x in inside if x.subnet_of(v)]):
            nxt = []
            for b in rest:
                if t.subnet_of(b):
                    nxt.extend(b.address_exclude(t))
                elif not b.subnet_of(t):
                    nxt.append(b)
            rest = nxt
        free_blocks.extend(sorted(rest))
    if total and free / total < 0.25:
        f("LOW", "LOW_HEADROOM", "plan", f"only {free / total:.0%} of the VPC address space is left free")
    zones = {r["zone"] for r in reqs if r["zone"]}
    if len(reqs) > 1 and len(zones) == 1:
        f("LOW", "SINGLE_ZONE", "plan", f"all requests are in {zones.pop()}; spread subnets over at least two zones")
    rank = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
    findings.sort(key=lambda x: rank[x["severity"]])
    return {"name": cfg.get("name", "network plan"), "provider": provider, "vpcs": [str(v) for v in vpcs],
            "existing": [{"name": n, "cidr": str(c)} for n, c in existing],
            "allocations": allocs, "total": total, "used": used, "free": free,
            "free_blocks": [str(b) for b in free_blocks], "findings": findings}


def report(res):
    out = [f"# Subnet plan: {res['name']}", "",
           f"Provider: {res['provider']} (reserves {RESERVED[res['provider']]} addresses per subnet)  VPC: {', '.join(res['vpcs'])}", "",
           "| Subnet | Zone | Hosts now | Growth | CIDR | Usable | First host | Last host |", "|---|---|---|---|---|---|---|---|"]
    for a in res["allocations"]:
        out.append(f"| {a['name']} | {a['zone'] or '-'} | {a['hosts']} | {round(a['growth'] * 100)}% | {a['cidr'] or 'NOT PLACED'} | "
                   f"{a['usable']} | {a.get('first', '-')} | {a.get('last', '-')} |")
    if res["existing"]:
        out += ["", "Existing subnets kept: " + ", ".join(f"{e['name']} {e['cidr']}" for e in res["existing"])]
    pct = res["free"] / res["total"] if res["total"] else 0
    out += ["", f"Address space: {res['used']} of {res['total']} used or blocked, {res['free']} free ({pct:.0%})",
            "Largest free blocks: " + (", ".join(sorted(res["free_blocks"], key=lambda c: ip.ip_network(c).prefixlen)[:6]) or "none"),
            "", "## Findings"]
    if not res["findings"]:
        out.append("- none")
    for x in res["findings"]:
        out.append(f"- **{x['severity']} {x['code']}** ({x['where']}): {x['message']}")
    return "\n".join(out) + "\n"


def self_test():
    cases = []

    def ok(name, cond):
        cases.append(bool(cond))
        print(("PASS " if cond else "FAIL ") + name)

    ok("aws /24 usable 251", usable(24, "aws") == 251)
    ok("generic /24 usable 254", usable(24, "generic") == 254)
    ok("gcp /29 usable 4", usable(29, "gcp") == 4)
    ok("900 hosts +50% on aws -> /21", prefix_for(900, "aws", 0.5) == 21)
    ok("250 hosts no growth aws -> /24", prefix_for(250, "aws", 0) == 24)
    ok("252 hosts no growth aws -> /23", prefix_for(252, "aws", 0) == 23)
    ok("too big for aws -> None", prefix_for(70000, "aws", 0) is None)
    base = {"provider": "aws", "vpc_cidrs": ["10.0.0.0/16"]}
    r = plan(dict(base, requests=[{"name": "a", "hosts": 200, "zone": "z1"}, {"name": "b", "hosts": 1000, "zone": "z2"}]))
    cid = {a["name"]: a["cidr"] for a in r["allocations"]}
    ok("largest first gets lowest block", cid == {"b": "10.0.0.0/21", "a": "10.0.8.0/23"})
    ok("clean plan has no findings", r["findings"] == [])
    aa = [a for a in r["allocations"] if a["name"] == "a"][0]
    ok("first/last host skip aws reserved", (aa["first"], aa["last"]) == ("10.0.8.4", "10.0.9.254"))
    g = plan({"provider": "gcp", "vpc_cidrs": ["10.0.0.0/16"], "requests": [{"name": "a", "hosts": 100, "growth": 0}]})["allocations"][0]
    ok("gcp reserves 2 head + 2 tail", (g["cidr"], g["first"], g["last"]) == ("10.0.0.0/25", "10.0.0.2", "10.0.0.125"))
    r = plan(dict(base, existing=[{"name": "old", "cidr": "10.0.0.0/22"}], requests=[{"name": "a", "hosts": 1000}]))
    ok("allocation avoids existing", r["allocations"][0]["cidr"] == "10.0.8.0/21")
    r = plan(dict(base, existing=[{"name": "x", "cidr": "10.0.0.0/24"}, {"name": "y", "cidr": "10.0.0.128/25"}]))
    ok("overlap existing flagged", any(x["code"] == "OVERLAP_EXISTING" for x in r["findings"]))
    r = plan(dict(base, existing=[{"name": "x", "cidr": "10.1.0.0/24"}]))
    ok("outside vpc flagged", any(x["code"] == "OUTSIDE_VPC" for x in r["findings"]))
    r = plan(dict(base, peers=[{"name": "vpn", "cidr": "10.0.4.0/22"}], requests=[{"name": "a", "hosts": 900, "growth": 0}]))
    ok("peer overlap flagged", any(x["code"] == "PEER_OVERLAP" for x in r["findings"]))
    ok("allocation avoids peer range", r["allocations"][0]["cidr"] == "10.0.0.0/22")
    r = plan({"provider": "aws", "vpc_cidrs": ["10.0.0.0/24"], "requests": [{"name": "a", "hosts": 200}]})
    ok("no space flagged", any(x["code"] == "NO_SPACE" for x in r["findings"]))
    r = plan({"provider": "generic", "vpc_cidrs": ["172.16.0.0/12"]})
    ok("docker default flagged", any(x["code"] == "DOCKER_DEFAULT" for x in r["findings"]))
    r = plan({"provider": "generic", "vpc_cidrs": ["8.8.0.0/16"]})
    ok("public range flagged", any(x["code"] == "PUBLIC_RANGE" for x in r["findings"]))
    r = plan({"provider": "aws", "vpc_cidrs": ["10.0.0.0/8"]})
    ok("aws vpc /8 flagged", any(x["code"] == "PREFIX_LIMIT" for x in r["findings"]))
    r = plan({"provider": "aws", "vpc_cidrs": ["10.0.0.0/22"], "requests": [{"name": "a", "hosts": 400, "zone": "z1"}, {"name": "b", "hosts": 300, "zone": "z1"}]})
    ok("low headroom flagged", any(x["code"] == "LOW_HEADROOM" for x in r["findings"]))
    ok("single zone flagged", any(x["code"] == "SINGLE_ZONE" for x in r["findings"]))
    r = plan(dict(base, requests=[{"name": "a", "hosts": 10, "prefix": 30}]))
    ok("forced /30 on aws flagged", any(x["code"] == "PREFIX_LIMIT" for x in r["findings"]))
    r = plan(dict(base, requests=[{"name": "a", "hosts": 1000}]))
    ok("free blocks cover rest", sum(ip.ip_network(b).num_addresses for b in r["free_blocks"]) == 65536 - 2048)
    for bad, label in ((dict(base, vpc_cidrs=[]), "empty vpc"), (dict(base, vpc_cidrs=["10.0.0.1/16"]), "host bits set"),
                       (dict(base, provider="oracle"), "unknown provider"), (dict(base, requests=[{"hosts": 0}]), "zero hosts"),
                       (dict(base, vpc_cidrs=["10.0.0.0/16", "10.0.128.0/17"]), "overlapping vpcs"),
                       (dict(base, vpc_cidrs=["fd00::/48"]), "ipv6")):
        try:
            plan(bad)
            ok(label + " rejected", False)
        except PlanError:
            ok(label + " rejected", True)
    print(f"{sum(cases)}/{len(cases)} passed")
    return 0 if all(cases) else 1


def main(argv=None):
    ap = argparse.ArgumentParser(description="Plan and check VPC subnet CIDR blocks.")
    ap.add_argument("file", nargs="?", help="network plan JSON")
    ap.add_argument("--json", action="store_true", help="print machine-readable JSON")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test:
        return self_test()
    if not a.file:
        ap.error("give a plan JSON file or --self-test")
    try:
        cfg = json.load(open(a.file, encoding="utf-8"))
        res = plan(cfg)
    except (OSError, json.JSONDecodeError, PlanError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(res, indent=2) if a.json else report(res), end="" if not a.json else "\n")
    return 1 if any(x["severity"] == "HIGH" for x in res["findings"]) else 0


if __name__ == "__main__":
    sys.exit(main())

---
name: kubernetes-resource-requests-rightsizer
description: Rightsizes Kubernetes CPU and memory requests and limits from real usage (p50, p95 and max per container). Recommends requests at p95 plus headroom, memory limits above the peak and no CPU limits, and estimates the cores, GiB and money freed. Flags OOM kills, memory above the request, CPU throttling, missing requests or memory limits, overprovisioning and short data windows. Includes a stdlib Python rightsizer with --self-test.
---

# Kubernetes Resource Requests Rightsizer

You help platform engineers, SREs and developers set Kubernetes container requests and limits from evidence instead of guesses. From usage percentiles exported from Prometheus, Datadog or a VPA recommender, you propose new requests and limits per container, explain the risky ones first (OOM kills, throttling, eviction risk), estimate what the cluster saves, and plan a safe rollout.

## Files in this skill

- `scripts/rightsize.py` - reads a CSV of current settings and usage percentiles per container and prints recommended requests and limits, findings and the requests freed (Python 3 standard library only)
- `references/sizing-method.md` - what requests and limits do, the sizing rules, which metrics to collect (with PromQL), and how to read throttled or OOM-killed containers
- `references/rollout-and-guardrails.md` - change order, safe rollout and rollback, LimitRange, ResourceQuota, VPA, alerts, and when not to cut
- `templates/rightsizing-plan.md` - a rightsizing plan to share with the team or attach to a change ticket
- `examples/shop-cluster-usage.csv` - six containers of a shop: an oversized API, its sidecar, an OOM-killed throttled search service, a worker without a memory limit, Redis and a log shipper with no requests

## Workflow

### 1. Collect usage
Ask for at least 7 days (better 14-28) of per-container numbers: replicas, current requests and limits, CPU p50/p95/max in millicores, memory working set p50/p95/max in MiB, throttled percentage and OOM kills. `references/sizing-method.md` has the PromQL for each. Ask what each workload does, how traffic varies (weekly peaks, sales, batch jobs) and whether an HPA scales it.

### 2. Run the rightsizer
```
python3 scripts/rightsize.py examples/shop-cluster-usage.csv --core-month 24 --gib-month 3.2
python3 scripts/rightsize.py usage.csv --headroom 25 --keep-cpu-limits --json
python3 scripts/rightsize.py --self-test
```
Prices are optional and per requested core or GiB per month. Exit code 1 means CRITICAL or HIGH findings, 2 means bad input.

### 3. Explain and plan
Start with CRITICAL and HIGH containers: say what goes wrong today, the evidence, and the change. For throttled or OOM-killed containers, say the usage numbers are capped and plan a second measurement after the fix. Then list the biggest savings (replicas multiply them). Check the cases in `references/rollout-and-guardrails.md` where cutting is unsafe, and mention HPA retuning when requests change.

### 4. Deliver
Fill `templates/rightsizing-plan.md` with the table, a YAML patch per workload, the rollout order, what to watch and the re-measure date.

## Example result
For `examples/shop-cluster-usage.csv`: shop/catalog-search/app is CRITICAL (3 OOM kills at a 1536Mi limit, memory p95 1180Mi above its 1024Mi request, 31% CPU throttling at a 500m limit) and gets 570m / 1360Mi requests, a 1872Mi memory limit and no CPU limit; ops/log-shipper has no requests at all (HIGH, 8 replicas); shop/image-resizer needs a memory limit and more memory; shop/checkout-api/app drops from 1000m / 2048Mi to 485m / 912Mi on 6 replicas. In total 1.76 cores and 4.06 GiB of requests are freed, about 55 a month at 24 per core and 3.2 per GiB.

## Rules

- Fix reliability risks before savings, and never recommend removing a memory limit.
- Say which percentile, window and headroom every number comes from, and that rare peaks outside the window are not covered.
- Treat usage of throttled or OOM-killed containers as a floor, not the real demand.
- Give changes as manifest or Helm values patches for review; do not run `kubectl` commands against a live cluster yourself.
- Report savings as requests freed until nodes are actually removed.
FILE:references/sizing-method.md
# Sizing Method

## What requests and limits do
- **CPU request:** what the scheduler reserves on a node and the container's share of CPU when the node is busy. Too low: the pod lands on crowded nodes and slows down under load. Too high: nodes look full while sitting idle, so you pay for empty cores.
- **CPU limit:** a hard cap enforced by CFS quota in 100 ms periods. A container that hits it is throttled, even when the node has idle CPU. Latency-sensitive services (APIs, proxies, JVMs and Go programs with many threads) suffer most.
- **Memory request:** what the scheduler reserves. Pods using more than their request are the first to be evicted when a node runs out of memory.
- **Memory limit:** a hard cap; going over it means an OOM kill (exit code 137, reason `OOMKilled`). Memory cannot be throttled.
- **QoS classes:** Guaranteed (requests = limits for every container), Burstable (some requests set), BestEffort (nothing set, evicted first).

## The rules used by `scripts/rightsize.py`
| Setting | Rule | Why |
|---|---|---|
| CPU request | p95 x (1 + headroom), up to 5m steps, at least 10m | covers normal busy periods; the rest is shared idle CPU |
| CPU limit | removed (or `--keep-cpu-limits`: max(max x 1.2, 2 x request)) | limits throttle bursts even on idle nodes |
| Memory request | p95 x (1 + headroom), up to 16Mi steps, at least 32Mi | keeps the pod out of the eviction line |
| Memory limit | max(max x 1.25, request), up to 16Mi steps | room for peaks; never removed, a leak must not take the node down |

Default headroom is 15%. Use 25-30% for spiky traffic, batch jobs or when the data window is short.

## Which numbers to collect
Per container, over at least 7 days (14-28 days catches weekly and month-end peaks):
- CPU in millicores: p50, p95 and max of `rate(container_cpu_usage_seconds_total[5m]) * 1000`
- Memory in MiB: p50, p95 and max of `container_memory_working_set_bytes / 2^20` (working set is what the OOM killer and eviction look at, not RSS or cache)
- Throttling: `container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total * 100`
- OOM kills: `kube_pod_container_status_last_terminated_reason{reason="OOMKilled"}` or restarts with exit code 137
- Current settings: `kubectl get deploy -o jsonpath` or `kube_pod_container_resource_requests` / `_limits`

## Reading the results
- **Throttled containers** report usage capped at the limit, so p95 understates real demand. Raise or remove the limit first, measure again for a few days, then size.
- **OOM kills** mean the max is unknown (the container died at the limit). Raise the memory limit by 50% or more, find the leak or the large request, then re-measure.
- **JVM, Node.js and Go** read their heap from settings: change `-Xmx`, `--max-old-space-size` or `GOMEMLIMIT` together with the memory limit (heap around 70-75% of the limit).
- **Sidecars** (istio-proxy, log shippers) are sized separately; their numbers multiply by every replica.
- **HPA on CPU** scales on usage divided by the request. Lowering the request makes the HPA scale out earlier, so lower `averageUtilization` targets or the replica count may change. Rightsize first, then retune the HPA.

## Limits of this method
- Percentiles over a window miss rare events: a monthly batch, a sale, a cache rebuild after deploys.
- Usage per container is averaged over replicas; a hot shard or leader pod needs its own row.
- Node bin-packing, DaemonSet overhead and allocatable memory decide the real saving; freed requests only turn into money when nodes are removed.
FILE:references/rollout-and-guardrails.md
# Rollout and Guardrails

## Change order
1. **Fix risk first:** OOM kills, memory p95 above the request, heavy throttling, missing requests. These cause incidents; savings can wait.
2. **Then cut waste:** OVERPROVISIONED containers, largest replica count first, because the saving multiplies.
3. **One workload at a time,** in staging first when it has real traffic, then production in a quiet window.

## Safe rollout steps
1. Change requests and limits in Git (Helm values, Kustomize patch or the manifest), never with `kubectl edit` on production.
2. Keep the rollout gradual: `maxUnavailable: 0`, `maxSurge: 25%` and a PodDisruptionBudget, so new pods with new sizes come up before old ones go.
3. Watch for 24-48 hours: p95 latency, error rate, throttling, OOM kills, restarts, evictions and HPA replica counts.
4. Rollback = revert the commit. Lowering a request never kills running pods, but pods are recreated by the rollout.

## Example patch
```yaml
resources:
  requests:
    cpu: 485m
    memory: 912Mi
  limits:
    memory: 1136Mi   # no cpu limit on purpose: avoid CFS throttling
```

## Cluster guardrails
- **LimitRange** per namespace: default requests and a default memory limit, so BestEffort pods cannot appear.
- **ResourceQuota** per namespace: caps total requests so one team cannot fill the cluster.
- **Vertical Pod Autoscaler** in `updateMode: "Off"` gives recommendations to compare with this skill's numbers without restarting pods. Do not let VPA (Auto) and an HPA on CPU or memory control the same workload.
- **Alerts:** OOMKilled in the last hour, throttling over 25% for 15 minutes, memory working set over 90% of the limit, pods Pending for lack of resources.

## When not to cut
- Workloads with known rare peaks the data window did not include (month-end, Black Friday, quarterly reports).
- Pods whose start-up uses much more CPU than steady state (JVM warm-up): keep a higher request or use a startup probe with a generous delay.
- Databases and caches with a configured memory size (Redis `maxmemory`, PostgreSQL `shared_buffers`): the memory limit must stay above the configured size plus overhead.
- Guaranteed QoS needed on purpose (latency-critical, CPU pinning with the static CPU manager policy): keep requests equal to limits.

## Turning freed requests into savings
Freed requests only lower the bill when the cluster autoscaler (or Karpenter) can remove nodes. Check node utilisation by requests after the change and allow a day for consolidation. Report savings as "requests freed" until node counts actually drop.
FILE:templates/rightsizing-plan.md
# Rightsizing Plan - <cluster / namespace> - <date>

**Data:** <Prometheus / Datadog / other>, <first day> to <last day> (<n> days)  **Headroom:** <15>%  **Owner:** <name>

## Risks to fix first
| Container | Finding | Evidence | Change | Due |
|---|---|---|---|---|
| | | | | |

## Requests and limits
| Container | Replicas | CPU req (now -> new) | CPU limit | Mem req (now -> new) | Mem limit (now -> new) | Status |
|---|---|---|---|---|---|---|
| | | | | | | |

## Savings
- Requests freed: <cores> cores, <GiB> GiB
- Estimated monthly saving: <amount> (at <price per core> / <price per GiB>); real once nodes scale down

## Rollout
| Step | Workload | Environment | Date | Watch | Rollback |
|---|---|---|---|---|---|
| 1 | | staging | | latency, OOM, throttling | revert commit |

## Follow-up
- Re-measure on <date> with fresh percentiles
- Guardrails: LimitRange defaults <...>, alerts <...>
FILE:examples/shop-cluster-usage.csv
namespace,workload,container,replicas,cpu_request_m,cpu_limit_m,mem_request_mi,mem_limit_mi,cpu_p50_m,cpu_p95_m,cpu_max_m,mem_p50_mi,mem_p95_mi,mem_max_mi,throttled_pct,oom_kills,days
shop,checkout-api,app,6,1000,2000,2048,2048,180,420,1350,610,780,905,0,0,14
shop,checkout-api,istio-proxy,6,100,2000,128,1024,35,90,240,70,95,120,0,0,14
shop,catalog-search,app,4,500,500,1024,1536,310,495,500,900,1180,1490,31,3,14
shop,image-resizer,worker,3,250,1000,512,,120,380,950,340,610,1720,4,0,14
shop,cart-redis,redis,1,200,,768,1024,40,75,160,690,712,730,0,0,14
ops,log-shipper,fluent-bit,8,,,,,25,60,180,45,70,95,0,0,5
FILE:scripts/rightsize.py
#!/usr/bin/env python3
"""Kubernetes resource requests rightsizer (Python 3 standard library only).

Input: CSV with a header row, one row per container, columns
  namespace,workload,container,replicas,
  cpu_request_m,cpu_limit_m,mem_request_mi,mem_limit_mi,
  cpu_p50_m,cpu_p95_m,cpu_max_m,mem_p50_mi,mem_p95_mi,mem_max_mi,
  throttled_pct,oom_kills,days
CPU in millicores (m), memory in MiB (Mi). Empty request/limit cells mean "not set".
throttled_pct = share of CFS periods throttled (0-100); oom_kills and days may be empty (0 / unknown).

Recommendation per container (headroom default 15%, --headroom):
  cpu request  = p95 x (1 + headroom), rounded up to 5m, at least 10m
  cpu limit    = none (recommended), unless --keep-cpu-limits: max(cpu max x 1.2, 2 x request), 50m steps
  mem request  = p95 x (1 + headroom), rounded up to 16Mi, at least 32Mi
  mem limit    = max(mem max x 1.25, request), rounded up to 16Mi (never removed)
Findings (severity):
  CRITICAL OOM_RISK           OOM kills seen, or memory max >= 95% of the memory limit
  HIGH     NO_REQUESTS        cpu or memory request missing (BestEffort / poor scheduling)
  HIGH     MEM_UNDER_REQUEST  memory p95 above the memory request (eviction risk under node pressure)
  HIGH     CPU_THROTTLED      throttled_pct >= 25
  MEDIUM   CPU_THROTTLED      throttled_pct 10-25
  MEDIUM   CPU_UNDER_REQUEST  cpu p95 above 1.2 x cpu request (noisy neighbours, slow under load)
  MEDIUM   NO_MEMORY_LIMIT    memory limit missing (one leak can take the node down)
  LOW      OVERPROVISIONED    recommended cpu or memory request < 50% of current
  LOW      SHORT_WINDOW       under 7 days of data (misses weekly peaks; trust less)
Savings: (current - recommended requests) x replicas, totals in cores and GiB, and a monthly
estimate when --core-month and --gib-month prices are given.
Usage:
  python3 rightsize.py usage.csv
  python3 rightsize.py usage.csv --headroom 25 --core-month 24 --gib-month 3.2 --json
  python3 rightsize.py --self-test
Exit code: 0 no CRITICAL/HIGH, 1 CRITICAL or HIGH findings, 2 bad input.
"""
import argparse
import csv
import io
import json
import math
import sys

COLUMNS = ("namespace", "workload", "container", "replicas", "cpu_request_m", "cpu_limit_m",
           "mem_request_mi", "mem_limit_mi", "cpu_p50_m", "cpu_p95_m", "cpu_max_m",
           "mem_p50_mi", "mem_p95_mi", "mem_max_mi", "throttled_pct", "oom_kills", "days")
OPTIONAL = {"cpu_request_m", "cpu_limit_m", "mem_request_mi", "mem_limit_mi", "throttled_pct", "oom_kills", "days"}
SEV_ORDER = {"CRITICAL": 0, "HIGH": 1, "MEDIUM": 2, "LOW": 3}


class DataError(ValueError):
    pass


def up(x, step):
    return int(math.ceil(x / step - 1e-9) * step)


def num(row, col, line):
    v = row.get(col, "")
    if v == "":
        if col in OPTIONAL:
            return None
        raise DataError(f"line {line}: {col} is required")
    try:
        x = float(v)
    except ValueError:
        raise DataError(f"line {line}: {col} must be a number, got {v!r}")
    if x < 0:
        raise DataError(f"line {line}: {col} must be >= 0")
    return x


def read_rows(text):
    rd = csv.DictReader(io.StringIO(text))
    if not rd.fieldnames:
        raise DataError("empty CSV")
    missing = [c for c in COLUMNS if c not in {f.strip() for f in rd.fieldnames}]
    if missing:
        raise DataError("CSV header is missing: " + ", ".join(missing))
    rows = []
    for n, raw in enumerate(rd, start=2):
        row = {k.strip(): (v or "").strip() for k, v in raw.items() if k}
        if not any(row.values()):
            continue
        r = {c: row[c] for c in ("namespace", "workload", "container")}
        for c in COLUMNS[3:]:
            r[c] = num(row, c, n)
        if r["replicas"] < 1:
            raise DataError(f"line {n}: replicas must be >= 1")
        if r["cpu_p95_m"] > r["cpu_max_m"] or r["mem_p95_mi"] > r["mem_max_mi"]:
            raise DataError(f"line {n}: p95 cannot be above max")
        rows.append(r)
    if not rows:
        raise DataError("no data rows")
    return rows


def analyze(r, headroom=0.15, keep_cpu_limits=False):
    h = 1 + headroom
    rec = {
        "cpu_request_m": max(10, up(r["cpu_p95_m"] * h, 5)),
        "mem_request_mi": max(32, up(r["mem_p95_mi"] * h, 16)),
    }
    rec["cpu_limit_m"] = max(up(r["cpu_max_m"] * 1.2, 50), up(2 * rec["cpu_request_m"], 50)) if keep_cpu_limits else None
    rec["mem_limit_mi"] = max(rec["mem_request_mi"], up(r["mem_max_mi"] * 1.25, 16))
    findings = []

    def f(sev, code, msg):
        findings.append({"severity": sev, "code": code, "message": msg})

    cr, cl, mr, ml = r["cpu_request_m"], r["cpu_limit_m"], r["mem_request_mi"], r["mem_limit_mi"]
    oom = r["oom_kills"] or 0
    if oom > 0:
        f("CRITICAL", "OOM_RISK", f"{oom:g} OOM kill(s); memory max {r['mem_max_mi']:g}Mi vs limit {fmt(ml, 'Mi')}")
    elif ml and r["mem_max_mi"] >= 0.95 * ml:
        f("CRITICAL", "OOM_RISK", f"memory max {r['mem_max_mi']:g}Mi is {r['mem_max_mi'] / ml:.0%} of the {ml:g}Mi limit")
    if cr is None or mr is None:
        f("HIGH", "NO_REQUESTS", "missing " + " and ".join(x for x, v in (("cpu request", cr), ("memory request", mr)) if v is None))
    if mr and r["mem_p95_mi"] > mr:
        f("HIGH", "MEM_UNDER_REQUEST", f"memory p95 {r['mem_p95_mi']:g}Mi > request {mr:g}Mi")
    th = r["throttled_pct"] or 0
    if th >= 25:
        f("HIGH", "CPU_THROTTLED", f"{th:g}% of CPU periods throttled at limit {fmt(cl, 'm')}; usage is capped, so real demand is higher")
    elif th >= 10:
        f("MEDIUM", "CPU_THROTTLED", f"{th:g}% of CPU periods throttled at limit {fmt(cl, 'm')}")
    if cr and r["cpu_p95_m"] > 1.2 * cr:
        f("MEDIUM", "CPU_UNDER_REQUEST", f"cpu p95 {r['cpu_p95_m']:g}m > 1.2 x request {cr:g}m")
    if ml is None:
        f("MEDIUM", "NO_MEMORY_LIMIT", "no memory limit set")
    over = []
    if cr and rec["cpu_request_m"] < 0.5 * cr:
        over.append(f"cpu {cr:g}m -> {rec['cpu_request_m']}m")
    if mr and rec["mem_request_mi"] < 0.5 * mr:
        over.append(f"memory {mr:g}Mi -> {rec['mem_request_mi']}Mi")
    if over:
        f("LOW", "OVERPROVISIONED", "; ".join(over))
    if r["days"] is not None and r["days"] < 7:
        f("LOW", "SHORT_WINDOW", f"only {r['days']:g} days of data")
    findings.sort(key=lambda x: SEV_ORDER[x["severity"]])
    status = findings[0]["severity"] if findings and findings[0]["severity"] != "LOW" else "OK"
    n = r["replicas"]
    d_cpu = ((cr or 0) - rec["cpu_request_m"]) * n
    d_mem = ((mr or 0) - rec["mem_request_mi"]) * n
    return {"id": f"{r['namespace']}/{r['workload']}/{r['container']}", "replicas": int(n), "current": {
        "cpu_request_m": cr, "cpu_limit_m": cl, "mem_request_mi": mr, "mem_limit_mi": ml},
        "recommended": rec, "delta_cpu_m": d_cpu, "delta_mem_mi": d_mem, "status": status, "findings": findings}


def run(text, headroom=0.15, keep_cpu_limits=False, core_month=None, gib_month=None):
    res = [analyze(r, headroom, keep_cpu_limits) for r in read_rows(text)]
    res.sort(key=lambda x: (SEV_ORDER.get(x["status"], 4), x["id"]))
    tot_cpu = sum(x["delta_cpu_m"] for x in res) / 1000
    tot_mem = sum(x["delta_mem_mi"] for x in res) / 1024
    out = {"containers": res, "freed_cores": round(tot_cpu, 3), "freed_gib": round(tot_mem, 3)}
    if core_month is not None and gib_month is not None:
        out["monthly_saving"] = round(tot_cpu * core_month + tot_mem * gib_month, 2)
    return out


def fmt(v, unit):
    return "none" if v is None else f"{v:g}{unit}"


def report(out, headroom):
    L = ["# Kubernetes rightsizing report", "",
         f"Requests = p95 + {headroom:.0%} headroom; CPU limits removed unless kept; memory limit = max x 1.25.", "",
         "| Container | Replicas | Status | CPU req | CPU limit | Mem req | Mem limit |",
         "|---|---|---|---|---|---|---|"]
    for x in out["containers"]:
        c, r = x["current"], x["recommended"]
        L.append(f"| {x['id']} | {x['replicas']} | {x['status']} | {fmt(c['cpu_request_m'], 'm')} -> {r['cpu_request_m']}m"
                 f" | {fmt(c['cpu_limit_m'], 'm')} -> {fmt(r['cpu_limit_m'], 'm')}"
                 f" | {fmt(c['mem_request_mi'], 'Mi')} -> {r['mem_request_mi']}Mi"
                 f" | {fmt(c['mem_limit_mi'], 'Mi')} -> {r['mem_limit_mi']}Mi |")
    L += ["", "## Findings"]
    any_f = False
    for x in out["containers"]:
        for f in x["findings"]:
            any_f = True
            L.append(f"- {f['severity']} {f['code']} {x['id']}: {f['message']}")
    if not any_f:
        L.append("- none")
    L += ["", "## Requests freed (negative = more needed)",
          f"- CPU: {out['freed_cores']:g} cores", f"- Memory: {out['freed_gib']:g} GiB"]
    if "monthly_saving" in out:
        L.append(f"- Estimated monthly saving: {out['monthly_saving']:,.2f}")
    return "\n".join(L) + "\n"


def self_test():
    cases = []

    def ok(name, cond):
        cases.append(bool(cond))
        print(("PASS " if cond else "FAIL ") + name)

    hdr = ",".join(COLUMNS) + "\n"

    def row(**kw):
        base = dict(namespace="ns", workload="w", container="c", replicas=1, cpu_request_m=500, cpu_limit_m=1000,
                    mem_request_mi=512, mem_limit_mi=1024, cpu_p50_m=100, cpu_p95_m=200, cpu_max_m=300,
                    mem_p50_mi=200, mem_p95_mi=300, mem_max_mi=400, throttled_pct=0, oom_kills=0, days=14)
        base.update(kw)
        return ",".join("" if base[c] is None else str(base[c]) for c in COLUMNS) + "\n"

    def one(**kw):
        return run(hdr + row(**kw))["containers"][0]

    def codes(x):
        return {(f["severity"], f["code"]) for f in x["findings"]}

    x = one()
    ok("cpu request = p95 x 1.15 rounded to 5m (230)", x["recommended"]["cpu_request_m"] == 230)
    ok("mem request = 300 x 1.15 = 345 -> 352Mi", x["recommended"]["mem_request_mi"] == 352)
    ok("mem limit = 400 x 1.25 = 500 -> 512Mi", x["recommended"]["mem_limit_mi"] == 512)
    ok("cpu limit removed by default", x["recommended"]["cpu_limit_m"] is None)
    ok("healthy container is OK", x["status"] == "OK")
    ok("headroom 0.25 changes cpu request (250)", run(hdr + row(), headroom=0.25)["containers"][0]["recommended"]["cpu_request_m"] == 250)
    k = run(hdr + row(), keep_cpu_limits=True)["containers"][0]["recommended"]["cpu_limit_m"]
    ok("kept cpu limit = max(360->400, 460->500) = 500", k == 500)
    ok("minimums 10m / 32Mi", one(cpu_p50_m=0, cpu_p95_m=1, cpu_max_m=2, mem_p50_mi=1, mem_p95_mi=2, mem_max_mi=3)["recommended"]
       == {"cpu_request_m": 10, "mem_request_mi": 32, "cpu_limit_m": None, "mem_limit_mi": 32})
    ok("OOM kills -> CRITICAL", ("CRITICAL", "OOM_RISK") in codes(one(oom_kills=2)) and one(oom_kills=2)["status"] == "CRITICAL")
    ok("max at 98% of limit -> OOM_RISK", ("CRITICAL", "OOM_RISK") in codes(one(mem_max_mi=1004, mem_p95_mi=500)))
    ok("missing requests -> HIGH NO_REQUESTS", ("HIGH", "NO_REQUESTS") in codes(one(cpu_request_m=None)))
    ok("mem p95 above request -> HIGH", ("HIGH", "MEM_UNDER_REQUEST") in codes(one(mem_request_mi=256)))
    ok("throttling 30% -> HIGH", ("HIGH", "CPU_THROTTLED") in codes(one(throttled_pct=30)))
    ok("throttling 12% -> MEDIUM", ("MEDIUM", "CPU_THROTTLED") in codes(one(throttled_pct=12)))
    ok("cpu p95 > 1.2 x request -> MEDIUM", ("MEDIUM", "CPU_UNDER_REQUEST") in codes(one(cpu_request_m=150)))
    ok("no memory limit -> MEDIUM", ("MEDIUM", "NO_MEMORY_LIMIT") in codes(one(mem_limit_mi=None)))
    big = one(cpu_request_m=2000, mem_request_mi=4096)
    ok("overprovisioned flagged LOW, status stays OK", ("LOW", "OVERPROVISIONED") in codes(big) and big["status"] == "OK")
    ok("short window flagged", ("LOW", "SHORT_WINDOW") in codes(one(days=3)))
    two = run(hdr + row(replicas=3) + row(workload="b", replicas=2, cpu_request_m=100, mem_request_mi=256),
              core_month=20, gib_month=4)
    # w: (500-230)*3=810m, (512-352)*3=480Mi ; b: (100-230)*2=-260m, (256-352)*2=-192Mi
    ok("freed cores = (810-260)/1000 = 0.55", two["freed_cores"] == 0.55)
    ok("freed GiB = (480-192)/1024 = 0.281", two["freed_gib"] == 0.281)
    ok("monthly saving = 0.55 x 20 + 0.28125 x 4 = 12.125", abs(two["monthly_saving"] - 12.125) < 0.01)
    ok("sorted by severity", two["containers"][0]["id"] == "ns/b/c")
    for bad, label in (("a,b\n1,2\n", "missing columns"), (hdr, "no rows"), (hdr + row(cpu_p95_m="x"), "bad number"),
                       (hdr + row(cpu_p95_m=900), "p95 above max"), (hdr + row(replicas=0), "zero replicas"),
                       (hdr + row(cpu_p95_m=None), "missing usage")):
        try:
            run(bad)
            ok(label + " rejected", False)
        except DataError:
            ok(label + " rejected", True)
    print(f"{sum(cases)}/{len(cases)} passed")
    return 0 if all(cases) else 1


def main(argv=None):
    ap = argparse.ArgumentParser(description="Rightsize Kubernetes container requests and limits from usage percentiles.")
    ap.add_argument("file", nargs="?", help="CSV with the columns listed in the module docstring")
    ap.add_argument("--headroom", type=float, default=15, help="percent added to p95 for requests (default 15)")
    ap.add_argument("--keep-cpu-limits", action="store_true", help="recommend CPU limits instead of removing them")
    ap.add_argument("--core-month", type=float, help="price of one requested core per month")
    ap.add_argument("--gib-month", type=float, help="price of one requested GiB of memory per month")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test:
        return self_test()
    if not a.file:
        ap.error("give a CSV file or --self-test")
    try:
        out = run(open(a.file, encoding="utf-8").read(), a.headroom / 100, a.keep_cpu_limits, a.core_month, a.gib_month)
    except (OSError, DataError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(out, indent=2) if a.json else report(out, a.headroom / 100), end="\n" if a.json else "")
    return 1 if any(x["status"] in ("CRITICAL", "HIGH") for x in out["containers"]) else 0


if __name__ == "__main__":
    sys.exit(main())

---
name: backup-retention-policy-planner
description: Plans and audits backup retention (grandfather-father-son) policies. Simulates hourly, daily, weekly, monthly and yearly keep rules like restic or borg, lists the restore points you really keep and the gaps between them, and estimates storage and cost now and in 12 months. Flags missed RPO, short history, legal retention gaps, no offsite or immutable copy, stale restore tests and job overlap. Includes a stdlib Python planner with --self-test.
---

# Backup Retention Policy Planner

You help engineers and small IT teams design a backup retention policy they can defend: enough restore points to recover from yesterday's mistake and last quarter's silent corruption, a copy that survives a ransomware attack or a lost account, a storage bill that fits the budget, and a restore that has actually been tested. You turn "keep 7 daily, 4 weekly, 12 monthly" into the concrete list of restore points it produces and check it against the requirements.

## Files in this skill

- `scripts/retention_plan.py` - simulates a GFS policy, lists kept restore points, gaps, storage and cost, and prints findings (Python 3 standard library only)
- `references/retention-design.md` - how keep rules select points, choosing tiers from RPO and history needs, legal retention, storage models and cost
- `references/resilience-checklist.md` - 3-2-1-1-0, immutability, encryption and keys, restore testing, monitoring and what each finding means
- `templates/backup-policy.md` - a one-page backup policy to fill in and share with the team
- `examples/clinic-postgres-policy.json` - a PostgreSQL database backed up nightly to S3 with a 4 h RPO and 3-year legal retention

## Workflow

### 1. Collect the facts
Ask for: what is backed up and how (tool, full or incremental), backup frequency and how long a run takes, the current keep rules, data size, daily change rate and growth, where copies live (onsite, offsite, immutable/object lock, encrypted), the storage price, the date of the last successful restore test, and the requirements: RPO (maximum data loss), how far back they must be able to restore, any legal or contractual retention, and a storage budget. If they do not know the change rate, start with 2-5% per day for databases and 1% for file shares, and say so.

### 2. Run the planner
```
python3 scripts/retention_plan.py examples/clinic-postgres-policy.json
python3 scripts/retention_plan.py policy.json --json
python3 scripts/retention_plan.py --self-test
```
Exit code 1 means HIGH findings, 2 means bad input.

### 3. Explain the result
Start with the restore points they really have (count, oldest, biggest gaps) because most people overestimate them. Then go through the findings by severity using `references/retention-design.md` and `references/resilience-checklist.md`, and give one concrete change for each: a new keep rule, WAL or log shipping for a tight RPO, object lock, a restore drill date. Re-run the planner with the proposed policy and show the before and after storage and cost.

### 4. Deliver
Fill `templates/backup-policy.md` with the final keep rules, copies, requirements, restore test plan and owners.

## Example result
For `examples/clinic-postgres-policy.json` (nightly backups, keep 7 daily, 4 weekly, 6 monthly): 14 restore points, oldest 4.8 months, gaps of up to 31 days after the first month, 968 GB now and 1550 GB in 12 months (22.27 to 35.66 USD/month). Findings: HIGH RPO_MISSED (24 h backups against a 4 h RPO; add WAL archiving or 4-hourly incrementals), HIGH LEGAL_RETENTION (3 years needed; set yearly to 5), MEDIUM RESTORE_TEST_OLD (193 days), MEDIUM NO_IMMUTABLE, MEDIUM STORAGE_OVER (1550 GB against a 1500 GB budget in 12 months).

## Rules

- A backup that has never been restored is not proven. Always ask for the last restore test and recommend a timed drill.
- Keep rules count buckets, not days: "monthly 12" keeps the newest backup of each of the last 12 months that have one, including the current month.
- Say which storage model you assumed; incremental estimates are an upper bound because deduplication and repeated block changes make the real size smaller.
- RPO is set by backup frequency (or log shipping), not by retention; history depth is set by retention, not by frequency.
- Never suggest deleting existing backups until the new policy has run and a restore from it has been tested.
FILE:references/retention-design.md
# Retention Design

## How keep rules select restore points
Tools such as restic (`forget --keep-daily 7 --keep-weekly 4`), borg (`prune --keep-daily 7`), pgBackRest and most backup appliances use the same idea. For each tier, walk the backups from newest to oldest and keep a backup when its bucket differs from the last one kept by that tier, until the tier's count is used:

| Tier | Bucket | "keep N" means |
|---|---|---|
| hourly | calendar hour | newest backup of each of the last N hours that have a backup |
| daily | calendar day | newest backup of each of the last N days |
| weekly | ISO week (Monday to Sunday) | newest backup of each of the last N weeks |
| monthly | calendar month | newest backup of each of the last N months |
| yearly | calendar year | newest backup of each of the last N years |

Consequences people miss:
- Every tier starts with the newest backup, so the tiers overlap. 7 daily + 4 weekly + 12 monthly is 20 points, not 23.
- The current week, month and year count as one bucket each. "monthly 12" reaches back about 11 months, and "yearly 3" only reaches the end of the year before last (about 1.8 years in late October).
- Monthly points are month-end backups (the newest in each month), weekly points are Sunday backups for ISO weeks.
- Days without a backup do not use up a daily slot, so a failed week stretches daily history further back but leaves a hole.

`scripts/retention_plan.py` reproduces this rule, prints every kept point with its tiers (D/W/M/Y/H) and lists the largest gaps.

## Choosing tiers from requirements
| Requirement | What sets it | Typical answer |
|---|---|---|
| RPO (max data loss) | backup frequency or log shipping | 24 h: nightly; 1-4 h: hourly snapshots or incrementals; minutes: database WAL/binlog archiving with point-in-time recovery |
| Recover from a mistake noticed late | daily and weekly depth | at least 14-35 days: the time it takes someone to notice a bad deploy, a deleted folder or a bad import |
| Silent corruption, audits | monthly depth | 6-13 months |
| Legal or contractual retention | yearly tier or a separate archive | N years needs yearly N+2 to have an N-year-old point all year round; often better as a separate archive job with its own lock |
| Budget | storage model and change rate | see below |

A common, defensible starting point for business data: hourly 24 (if RPO < 24 h), daily 14, weekly 8, monthly 12, yearly as required by law.

Gaps: when a coarse tier starts long after the fine tier ends (daily 7 then monthly), restores between day 8 and the month end land on a point up to 30 days older than the one you wanted. Add weekly points or more daily points (`TIER_GAP`).

## Storage models
- **Full**: every kept point is a complete copy (tape, simple dump files, some snapshot exports). Storage = points x full size.
- **Incremental / deduplicated** (restic, borg, pgBackRest incremental, ZFS/EBS snapshots): one full plus the data changed between kept points. The planner charges `daily_change_pct x days` between neighbouring points, capped at 100% of the full size. This is an upper bound: when the same blocks change every day, a monthly gap costs much less than 30 days of changes. Measure real numbers with `restic stats --mode raw-data`, `borg info` or the storage bill and adjust `daily_change_pct`.
- Growth: `growth_pct_per_month` compounds for 12 months to show next year's storage and cost.

Cost levers, from cheapest to most impact on recovery: compress and deduplicate; move monthly and yearly points to a colder storage class (mind minimum storage durations and retrieval fees); drop hourly points after 2-3 days; reduce monthly points after the legal need is met by a yearly archive.

## Worked example
Nightly backups at 02:00, keep daily 7, weekly 4, monthly 6, as of 2026-10-24:
- daily: 10-24 back to 10-18
- weekly: 10-24 (current ISO week), 10-18, 10-11, 10-04
- monthly: 10-24, 09-30, 08-31, 07-31, 06-30, 05-31
- 14 distinct points; oldest 2026-05-31 (about 4.8 months); the gaps after the first month are 30-31 days.
FILE:references/resilience-checklist.md
# Resilience Checklist

## 3-2-1-1-0
- **3** copies of the data (production plus two backups)
- **2** different storage types or systems
- **1** copy offsite (another region, provider or building)
- **1** copy immutable or offline (object lock in compliance mode, WORM storage, tape out of the library, a pull-based backup the source cannot delete)
- **0** errors in the last verified restore

## What each finding means and how to fix it
| Code | Severity | Why it matters | Typical fix |
|---|---|---|---|
| RPO_MISSED | HIGH | A failure just before the next backup loses more data than the business accepted | Shorter interval, hourly incrementals, or continuous log shipping (PostgreSQL WAL archiving with pgBackRest/WAL-G, MySQL binlog, SQL Server log backups) |
| HISTORY_SHORT | HIGH | Problems noticed late (bad import, deleted folder, slow corruption) cannot be undone | More daily/weekly points; at least the time it takes to notice a problem plus a margin |
| LEGAL_RETENTION | HIGH | Contract, tax or health-record rules can require years of history | Yearly tier of N+2 or a separate yearly archive job with object lock retention set to the legal period |
| NO_OFFSITE | HIGH | Fire, region outage or a compromised account removes production and backups together | Replicate to another region/provider with separate credentials |
| NEVER_RESTORED | HIGH | Backups fail silently: missing files, wrong permissions, lost encryption keys, untested runbooks | Restore drill now; record the time it took (actual RTO) |
| OVERLAP | HIGH | Jobs collide or get skipped, so the real interval is longer than planned | Incremental backups, faster storage, or a longer interval with log shipping |
| RESTORE_TEST_OLD | MEDIUM | Systems change; last spring's test says little about today | Quarterly drills; monthly automated restore-and-checksum for critical data |
| NO_IMMUTABLE | MEDIUM | Ransomware and attackers delete backups first | S3 Object Lock / Azure immutable blob / GCS bucket lock, or an offline copy |
| NOT_ENCRYPTED | MEDIUM | An offsite copy leaks all data if the bucket or tape is exposed | Client-side encryption; keys stored outside the backup system and escrowed |
| STORAGE_OVER | MEDIUM | The budget forces someone to cut retention in a hurry later | Colder storage for old tiers, fewer monthly points, measure the real change rate |
| TIER_GAP | LOW | Restores in the gap land on a much older point than wanted | Add weekly points or extend daily points |
| TIGHT_WINDOW | LOW | Data growth will push the job into overlap | Plan incremental or parallel backups before it happens |

## Restore drill (one page)
1. Pick a random restore point, not the newest.
2. Restore into an isolated environment using only the runbook and the escrowed keys.
3. Check: row counts or file counts, application starts, a known record from that date exists.
4. Record start and end time (actual RTO), problems and runbook fixes.
5. Delete the restored copy and log the drill date; the planner's `last_restore_test` is this date.

## Monitoring
- Alert when the newest backup is older than 1.5 x the interval.
- Alert on failed or partial jobs, not only on missing ones.
- Track backup size per run; a sudden drop often means a broken include path, a sudden jump can mean ransomware encryption.
- Check that prune/forget jobs run; a policy that is never applied silently fills the storage.

## Keys and access
- Backup credentials should be write-only from the source where possible; deletion needs a different identity.
- Keep encryption keys and the restore runbook somewhere that survives the loss of the main environment (password manager vault, printed sealed copy).
FILE:templates/backup-policy.md
# Backup Policy - <system name>

**Owner:** <team / person>  **Reviewed:** <date>  **Next review:** <date + 6 months>

## Scope
- Data: <databases, buckets, file shares, configs>
- Excluded: <caches, rebuildable artifacts>

## Requirements
| Item | Target | Source |
|---|---|---|
| RPO (max data loss) | <4 h> | <business owner> |
| RTO (max restore time) | <2 h> | <business owner> |
| Minimum history | <35 days> | <ops> |
| Legal / contractual retention | <3 years> | <contract, regulation> |
| Storage budget | <1500 GB / 40 USD per month> | <finance> |

## Schedule and keep rules
| Job | Frequency | Method | Keep rules |
|---|---|---|---|
| <full/incremental> | <daily 02:00> | <pgBackRest incr to S3> | <daily 14, weekly 8, monthly 12, yearly 5> |
| <log shipping> | <continuous> | <WAL archive> | <14 days> |

Restore points produced (from `scripts/retention_plan.py`): <count>, oldest <age>, largest gap <days>.

## Copies (3-2-1-1-0)
| Copy | Location | Immutable | Encrypted | Credentials |
|---|---|---|---|---|
| Primary backup | <region A bucket> | <no> | <yes> | <backup-writer role> |
| Offsite | <region B / provider B> | <object lock 35 d> | <yes> | <separate account> |

## Storage and cost
- Now: <GB> / <cost per month>  - In 12 months: <GB> / <cost per month>

## Restore testing
- Cadence: <quarterly drill, monthly automated check>
- Last test: <date>, result <ok / issues>, actual RTO <time>
- Runbook: <link>  - Keys escrowed at: <location>

## Monitoring and alerts
- <newest backup older than 36 h -> page on-call>
- <job failed / size change > 30% -> ticket>

## Open findings
- <severity code: action, owner, due date>
FILE:examples/clinic-postgres-policy.json
{
  "name": "clinic-booking PostgreSQL (pgBackRest to S3)",
  "today": "2026-10-24",
  "backup_interval_hours": 24,
  "backup_duration_hours": 2.5,
  "policy": {"daily": 7, "weekly": 4, "monthly": 6, "yearly": 0},
  "size": {"full_gb": 180, "daily_change_pct": 3, "growth_pct_per_month": 4, "storage_model": "incremental"},
  "requirements": {
    "rpo_hours": 4,
    "min_history_days": 90,
    "legal_retention_years": 3,
    "max_storage_gb": 1500,
    "restore_test_max_age_days": 90
  },
  "copies": {"onsite": false, "offsite": true, "immutable": false, "encrypted": true},
  "price_per_gb_month": 0.023,
  "currency": "USD",
  "last_restore_test": "2026-04-14"
}
FILE:scripts/retention_plan.py
#!/usr/bin/env python3
"""Backup retention policy planner (Python 3 standard library only).

Simulates a grandfather-father-son (GFS) retention policy the way restic/borg "keep" rules
work, then reports the restore points you really keep, the gaps between them, storage and
cost, and the gaps against your requirements (RPO, minimum history, legal retention, 3-2-1).

Input JSON:
  {"name": "orders-db", "today": "2026-10-24",
   "backup_interval_hours": 24, "backup_duration_hours": 1.5,
   "policy": {"hourly": 0, "daily": 7, "weekly": 4, "monthly": 12, "yearly": 3},
   "size": {"full_gb": 120, "daily_change_pct": 4, "growth_pct_per_month": 3,
            "storage_model": "incremental"},          # or "full" (every point is a full copy)
   "requirements": {"rpo_hours": 24, "min_history_days": 35, "legal_retention_years": 0,
                    "max_storage_gb": 0, "restore_test_max_age_days": 90},
   "copies": {"onsite": true, "offsite": true, "immutable": false, "encrypted": true},
   "price_per_gb_month": 0.023, "currency": "USD",
   "last_restore_test": "2026-03-02"}                 # or null if never tested
Selection rule per tier (newest backup first): keep a backup if its bucket (hour, day, ISO week,
month, year) differs from the previous kept bucket of that tier, until the tier count is used.
A backup kept by several tiers is stored once.
Storage: "full" = points x full size; "incremental" = one full + each later point's changed data
(daily_change_pct x days since the previous kept point, capped at 100%).
Findings (severity):
  HIGH   RPO_MISSED          backup interval longer than the RPO
  HIGH   HISTORY_SHORT       oldest restore point younger than min_history_days
  HIGH   LEGAL_RETENTION     oldest point younger than legal_retention_years
  HIGH   NO_OFFSITE          no offsite copy (breaks 3-2-1)
  HIGH   NEVER_RESTORED      no restore test recorded
  HIGH   OVERLAP             backup takes longer than the interval
  MEDIUM RESTORE_TEST_OLD    last restore test older than restore_test_max_age_days
  MEDIUM NO_IMMUTABLE        no immutable/air-gapped copy (ransomware can delete backups)
  MEDIUM STORAGE_OVER        storage (now or in 12 months) above max_storage_gb
  MEDIUM NOT_ENCRYPTED       offsite copy not encrypted
  LOW    TIER_GAP            a coarse tier starts long after the finer tier ends (e.g. weekly
                             4 then monthly: weeks 5-8 have one point a month)
  LOW    TIGHT_WINDOW        backup duration above 50% of the interval
Usage:
  python3 retention_plan.py policy.json
  python3 retention_plan.py policy.json --json
  python3 retention_plan.py --self-test
Exit code: 0 no HIGH findings, 1 HIGH findings, 2 bad input.
"""
import argparse
import datetime as dt
import json
import sys

TIERS = ["hourly", "daily", "weekly", "monthly", "yearly"]
SEV_ORDER = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}


class DataError(ValueError):
    pass


def bucket(tier, t):
    if tier == "hourly":
        return t.strftime("%Y-%m-%d %H")
    if tier == "daily":
        return t.date().isoformat()
    if tier == "weekly":
        y, w, _ = t.isocalendar()
        return f"{y}-W{w:02d}"
    if tier == "monthly":
        return t.strftime("%Y-%m")
    return str(t.year)


def simulate(policy, interval_h, today):
    """Return (kept list of (datetime, [tiers])) newest first."""
    for k in policy:
        if k not in TIERS:
            raise DataError(f"unknown policy tier '{k}' (use {', '.join(TIERS)})")
        if not isinstance(policy[k], int) or policy[k] < 0:
            raise DataError(f"policy.{k} must be a whole number >= 0")
    if sum(policy.get(t, 0) for t in TIERS) == 0:
        raise DataError("policy keeps nothing")
    if not interval_h or interval_h <= 0:
        raise DataError("backup_interval_hours must be > 0")
    span_days = max(policy.get("hourly", 0) / 24, policy.get("daily", 0), policy.get("weekly", 0) * 7,
                    policy.get("monthly", 0) * 31, policy.get("yearly", 0) * 366) + 400
    newest = dt.datetime.combine(today, dt.time(2, 0))
    step = dt.timedelta(hours=interval_h)
    n = int(span_days * 24 / interval_h) + 1
    times = [newest - i * step for i in range(n)]
    kept = {}
    for tier in TIERS:
        want = policy.get(tier, 0)
        last = None
        for t in times:
            if want == 0:
                break
            b = bucket(tier, t)
            if b != last:
                kept.setdefault(t, []).append(tier)
                last = b
                want -= 1
    return sorted(kept.items(), key=lambda kv: kv[0], reverse=True)


def storage_gb(kept, size):
    full = float(size.get("full_gb", 0))
    model = size.get("storage_model", "incremental")
    if model not in ("incremental", "full"):
        raise DataError("size.storage_model must be 'incremental' or 'full'")
    if model == "full":
        return full * len(kept)
    change = float(size.get("daily_change_pct", 5)) / 100
    total = full
    for (newer, _), (older, _) in zip(kept, kept[1:]):
        days = (newer - older).total_seconds() / 86400
        total += full * min(1.0, change * days)
    return total


def plan(cfg, today=None):
    try:
        today = today or dt.date.fromisoformat(cfg.get("today") or dt.date.today().isoformat())
    except ValueError:
        raise DataError("today must be YYYY-MM-DD")
    policy = cfg.get("policy") or {}
    interval = float(cfg.get("backup_interval_hours", 24))
    kept = simulate(policy, interval, today)
    newest = kept[0][0]
    ages = [(newest - t).total_seconds() / 86400 for t, _ in kept]
    oldest_days = ages[-1]
    size = cfg.get("size") or {}
    req = cfg.get("requirements") or {}
    copies = cfg.get("copies") or {}
    price = float(cfg.get("price_per_gb_month", 0))
    gb = storage_gb(kept, size)
    growth = float(size.get("growth_pct_per_month", 0)) / 100
    gb12 = gb * (1 + growth) ** 12
    tiers = []
    for tier in TIERS:
        pts = [(t, a) for (t, tl), a in zip(kept, ages) if tier in tl]
        if pts:
            tiers.append({"tier": tier, "count": len(pts), "newest_age_days": round(pts[0][1], 1),
                          "oldest_age_days": round(pts[-1][1], 1)})
    gaps = []
    for (t1, _), (t2, _), a in zip(kept, kept[1:], ages[1:]):
        gaps.append({"from": t2.date().isoformat(), "to": t1.date().isoformat(),
                     "gap_days": round((t1 - t2).total_seconds() / 86400, 1), "age_days": round(a, 1)})
    findings = []

    def add(sev, code, msg):
        findings.append({"severity": sev, "code": code, "message": msg})

    rpo = req.get("rpo_hours")
    if rpo and interval > rpo:
        add("HIGH", "RPO_MISSED", f"backups every {interval:g} h but RPO is {rpo:g} h; up to {interval:g} h of data can be lost")
    mh = req.get("min_history_days")
    if mh and oldest_days < mh:
        add("HIGH", "HISTORY_SHORT", f"oldest restore point is {oldest_days:.0f} days old; requirement is {mh} days")
    ly = req.get("legal_retention_years")
    if ly and oldest_days < ly * 365:
        add("HIGH", "LEGAL_RETENTION", f"oldest point {oldest_days / 365:.1f} years; legal retention needs {ly} years; yearly N also counts the current year, so set yearly >= {ly + 2} to cover it all year round")
    if not copies.get("offsite"):
        add("HIGH", "NO_OFFSITE", "no offsite copy; a site loss, account compromise or region outage takes every backup")
    dur = cfg.get("backup_duration_hours")
    if dur and dur > interval:
        add("HIGH", "OVERLAP", f"backup takes {dur:g} h but runs every {interval:g} h; jobs overlap or get skipped")
    elif dur and dur > interval / 2:
        add("LOW", "TIGHT_WINDOW", f"backup takes {dur:g} h of a {interval:g} h interval; growth will push it into overlap")
    lrt = cfg.get("last_restore_test")
    max_age = req.get("restore_test_max_age_days", 90)
    if not lrt:
        add("HIGH", "NEVER_RESTORED", "no restore test recorded; an untested backup is a hope, not a backup")
    else:
        age = (today - dt.date.fromisoformat(lrt)).days
        if age > max_age:
            add("MEDIUM", "RESTORE_TEST_OLD", f"last restore test {age} days ago (limit {max_age}); schedule a timed restore")
    if not copies.get("immutable"):
        add("MEDIUM", "NO_IMMUTABLE", "no immutable or air-gapped copy; ransomware or a stolen admin key can delete every backup")
    if copies.get("offsite") and copies.get("encrypted") is False:
        add("MEDIUM", "NOT_ENCRYPTED", "offsite copy is not encrypted at rest")
    mx = req.get("max_storage_gb")
    if mx and max(gb, gb12) > mx:
        add("MEDIUM", "STORAGE_OVER", f"storage {gb:.0f} GB now, {gb12:.0f} GB in 12 months; budget {mx} GB")
    order = [t for t in TIERS if policy.get(t, 0) > 0]
    span = {"hourly": 1 / 24, "daily": 1, "weekly": 7, "monthly": 30.4, "yearly": 365}
    for fine, coarse in zip(order, order[1:]):
        fine_end = policy[fine] * span[fine]
        if span[coarse] >= 4 * fine_end and fine != "hourly":
            add("LOW", "TIER_GAP", f"{fine} points cover ~{fine_end:.0f} days, then only one {coarse} point per {span[coarse]:.0f} days")
    findings.sort(key=lambda f: SEV_ORDER[f["severity"]])
    return {
        "name": cfg.get("name", "backup set"), "today": today.isoformat(), "interval_hours": interval,
        "restore_points": len(kept), "oldest_age_days": round(oldest_days, 1),
        "tiers": tiers, "largest_gaps": sorted(gaps, key=lambda g: -g["gap_days"])[:5],
        "storage_gb": round(gb, 1), "storage_gb_12m": round(gb12, 1),
        "monthly_cost": round(gb * price, 2), "monthly_cost_12m": round(gb12 * price, 2),
        "currency": cfg.get("currency", "USD"), "storage_model": size.get("storage_model", "incremental"),
        "kept": [{"time": t.strftime("%Y-%m-%d %H:%M"), "tiers": tl} for t, tl in kept],
        "findings": findings,
    }


def age_label(d):
    if d < 2:
        return f"{d * 24:.0f} h"
    if d < 60:
        return f"{d:.0f} d"
    if d < 730:
        return f"{d / 30.4:.1f} mo"
    return f"{d / 365:.1f} y"


def render(r):
    out = [f"# Backup retention plan: {r['name']} (as of {r['today']})", "",
           f"Backups every {r['interval_hours']:g} h -> {r['restore_points']} restore points kept, oldest {age_label(r['oldest_age_days'])} old",
           f"Storage ({r['storage_model']}): {r['storage_gb']:.0f} GB now, {r['storage_gb_12m']:.0f} GB in 12 months"
           f" -> {r['monthly_cost']:.2f} -> {r['monthly_cost_12m']:.2f} {r['currency']}/month", "",
           "| Tier | Points | Newest age | Oldest age |", "|---|---|---|---|"]
    for t in r["tiers"]:
        out.append(f"| {t['tier']} | {t['count']} | {age_label(t['newest_age_days'])} | {age_label(t['oldest_age_days'])} |")
    out += ["", "## Largest gaps between restore points", "| Older point | Newer point | Gap | Age of older point |", "|---|---|---|---|"]
    for g in r["largest_gaps"]:
        out.append(f"| {g['from']} | {g['to']} | {g['gap_days']:g} d | {age_label(g['age_days'])} |")
    out += ["", "## Restore points kept"]
    out.append(", ".join(f"{k['time'][:10]} ({'/'.join(x[0].upper() for x in k['tiers'])})" for k in r["kept"]))
    out += ["", "## Findings"]
    if not r["findings"]:
        out.append("- none")
    for f in r["findings"]:
        out.append(f"- {f['severity']} {f['code']}: {f['message']}")
    return "\n".join(out) + "\n"


def self_test():
    ok = 0
    tests = []

    def check(name, cond):
        nonlocal ok
        tests.append(name)
        print(("PASS " if cond else "FAIL ") + name)
        ok += bool(cond)

    today = dt.date(2026, 10, 24)
    base = {"today": "2026-10-24", "backup_interval_hours": 24, "policy": {"daily": 7, "weekly": 4, "monthly": 12},
            "size": {"full_gb": 100, "daily_change_pct": 2}, "requirements": {"rpo_hours": 24, "min_history_days": 35},
            "copies": {"offsite": True, "immutable": True, "encrypted": True}, "last_restore_test": "2026-10-01"}
    r = plan(base, today)
    # daily 7 = 7 newest days; weekly 4 overlaps with dailies; monthly 12
    check("daily 7 + weekly 4 + monthly 12 -> 20 distinct points", r["restore_points"] == 20)
    check("clean policy has no findings", r["findings"] == [])
    check("oldest point is the 2025-11-30 monthly", r["kept"][-1]["time"].startswith("2025-11-30"))
    k = simulate({"daily": 3}, 24, today)
    check("daily 3 keeps 3 newest days", [t.date().day for t, _ in k] == [24, 23, 22])
    k = simulate({"hourly": 6}, 1, today)
    check("hourly 6 keeps 6 points one hour apart", len(k) == 6 and (k[0][0] - k[5][0]) == dt.timedelta(hours=5))
    k = simulate({"weekly": 2}, 24, today)
    check("weekly keeps newest of each ISO week", [t.date().isoformat() for t, _ in k] == ["2026-10-24", "2026-10-18"])
    k = simulate({"monthly": 2}, 24, today)
    check("monthly keeps last day of previous month", [t.date().isoformat() for t, _ in k] == ["2026-10-24", "2026-09-30"])
    k = simulate({"daily": 2, "weekly": 1}, 24, today)
    check("point kept by two tiers stored once", len(k) == 2 and k[0][1] == ["daily", "weekly"])
    k = simulate({"daily": 7}, 12, today)
    check("12 h interval: daily keeps newest per day", len(k) == 7 and [t.hour for t, _ in k[:2]] == [2, 14])
    check("full storage = points x full", storage_gb([(dt.datetime(2026, 1, 2), []), (dt.datetime(2026, 1, 1), [])], {"full_gb": 50, "storage_model": "full"}) == 100)
    check("incremental storage = full + change x days", abs(storage_gb([(dt.datetime(2026, 1, 11), []), (dt.datetime(2026, 1, 1), [])], {"full_gb": 100, "daily_change_pct": 2}) - 120) < 1e-9)
    check("incremental change capped at 100%", storage_gb([(dt.datetime(2026, 6, 1), []), (dt.datetime(2026, 1, 1), [])], {"full_gb": 10, "daily_change_pct": 5}) == 20)

    def codes(**over):
        c = json.loads(json.dumps(base))
        for key, val in over.items():
            if key != "policy" and isinstance(val, dict) and isinstance(c.get(key), dict):
                c[key].update(val)
            else:
                c[key] = val
        return {f["code"] for f in plan(c, today)["findings"]}

    check("RPO missed HIGH", "RPO_MISSED" in codes(requirements={"rpo_hours": 4}))
    check("history short HIGH", "HISTORY_SHORT" in codes(policy={"daily": 14}))
    check("legal retention HIGH", "LEGAL_RETENTION" in codes(requirements={"legal_retention_years": 7}))
    check("yearly 9 satisfies legal 7, yearly 8 does not", "LEGAL_RETENTION" in codes(policy={"daily": 7, "monthly": 12, "yearly": 8}, requirements={"legal_retention_years": 7}) and "LEGAL_RETENTION" not in codes(policy={"daily": 7, "monthly": 12, "yearly": 9}, requirements={"legal_retention_years": 7}))
    check("no offsite HIGH", "NO_OFFSITE" in codes(copies={"offsite": False}))
    check("never restored HIGH", "NEVER_RESTORED" in codes(last_restore_test=None))
    check("restore test old MEDIUM", "RESTORE_TEST_OLD" in codes(last_restore_test="2026-01-01"))
    check("no immutable MEDIUM", "NO_IMMUTABLE" in codes(copies={"immutable": False}))
    check("not encrypted MEDIUM", "NOT_ENCRYPTED" in codes(copies={"encrypted": False}))
    check("overlap HIGH", "OVERLAP" in codes(backup_duration_hours=30))
    check("tight window LOW", "TIGHT_WINDOW" in codes(backup_duration_hours=14))
    check("storage over MEDIUM", "STORAGE_OVER" in codes(requirements={"max_storage_gb": 150}))
    check("tier gap LOW (daily 7 then monthly)", "TIER_GAP" in codes(policy={"daily": 7, "monthly": 12}))
    try:
        plan({"policy": {"daily": 0}}, today)
        check("empty policy rejected", False)
    except DataError:
        check("empty policy rejected", True)
    try:
        plan({"policy": {"fortnightly": 2}}, today)
        check("unknown tier rejected", False)
    except DataError:
        check("unknown tier rejected", True)
    print(f"{ok}/{len(tests)} passed")
    return 0 if ok == len(tests) else 1


def main():
    ap = argparse.ArgumentParser(description="Backup retention (GFS) policy planner")
    ap.add_argument("policy", nargs="?", help="policy JSON file")
    ap.add_argument("--json", action="store_true", help="print JSON instead of Markdown")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args()
    if a.self_test:
        return self_test()
    if not a.policy:
        ap.error("policy file required")
    try:
        with open(a.policy, encoding="utf-8") as fh:
            cfg = json.load(fh)
        r = plan(cfg)
    except (OSError, json.JSONDecodeError, DataError, ValueError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(r, indent=2) if a.json else render(r), end="" if not a.json else "\n")
    return 1 if any(f["severity"] == "HIGH" for f in r["findings"]) else 0


if __name__ == "__main__":
    sys.exit(main())

---
name: nginx-config-security-reviewer
description: Reviews nginx.conf and site configs for security problems such as TLS 1.0/1.1, weak ciphers, missing HSTS and security headers, the add_header inheritance trap, alias off-by-slash path traversal, autoindex, unlimited body size, $uri in redirects, HTTP without an HTTPS redirect, open admin paths and logins without rate limits. Gives a fixed config. Includes a stdlib Python checker with --self-test.
---

# Nginx Config Security Reviewer

You review nginx configurations the way a careful ops engineer would before a security audit: find the settings that expose data or weaken TLS, explain each one in plain words, and hand back a corrected config that still serves the same routes. You never guess at files you have not seen, and you never tell the user to reload without `nginx -t`.

## Files in this skill

- `scripts/nginx_review.py` - parses nginx config files into blocks and prints findings with severity, line and fix (Python 3 standard library only)
- `references/hardening-checklist.md` - every check the script runs, why it matters and the fix, plus items to check by hand
- `references/tls-and-headers.md` - TLS protocol and cipher settings, HSTS rollout, security headers and how add_header inheritance works
- `templates/review-report.md` - the report you hand back: summary, findings table, config diff, test commands
- `examples/shop-site.conf` - a small shop config from 2019 with 18 findings (3 HIGH)
- `examples/shop-site-fixed.conf` - the same site after the fixes (no findings apart from the include note)

## Workflow

### 1. Collect the full config
Ask for the output of `sudo nginx -T` (it prints the main file and every included file), or for nginx.conf plus the relevant sites-enabled files. Ask what the site does (static, reverse proxy, PHP), which paths must stay public, the nginx version (`nginx -v`) and whether a CDN or load balancer terminates TLS in front of it. Tell the user to remove private keys or secrets before pasting; certificate paths are fine.

### 2. Run the checker
```
python3 scripts/nginx_review.py examples/shop-site.conf
python3 scripts/nginx_review.py /etc/nginx/nginx.conf /etc/nginx/sites-enabled/shop --json
python3 scripts/nginx_review.py --self-test
```
Exit code 1 means HIGH findings, 2 means the file could not be read or parsed (the error names the line).

### 3. Confirm and add manual checks
Read each finding against the user's context: a CDN in front changes TLS findings, an internal-only admin route may already be firewalled, a `return 404` may already hide a path. Then go through the manual items in `references/hardening-checklist.md` (included files, upstream TLS verification, `if` blocks, PHP handlers, file permissions). Drop false positives and say why.

### 4. Deliver
Fill `templates/review-report.md`: HIGH items first, a minimal diff of the config, and the test steps. Explain the add_header inheritance rule whenever ADD_HEADER_INHERITANCE appears, because it surprises most people. Introduce HSTS with a short max-age first if the site has never used it.

## Example result
For `examples/shop-site.conf`: 3 HIGH (TLS 1.0/1.1 enabled, RC4 and 3DES ciphers, `location /images` with `alias .../images/` allowing `/images../` traversal), 6 MEDIUM (unlimited body size, port 80 serving content instead of redirecting, autoindex on /downloads/, /api/ dropping HSTS and nosniff because it sets its own add_header, unprotected /admin/, `$uri` in a redirect), 8 LOW (server_tokens, dotfiles, missing CSP/Referrer-Policy/X-Frame-Options, proxy headers, no rate limit on /api/auth/login) and 1 INFO. `examples/shop-site-fixed.conf` shows the fixes and passes `nginx -t`.

## Rules

- Quote the exact line and the exact replacement; keep changes minimal and route-preserving.
- Always end with `sudo nginx -t && sudo systemctl reload nginx` and a curl check; never suggest a restart that drops connections when a reload works.
- Do not invent directives: check names against the nginx docs for the user's version (for example `http2 on;` needs 1.25.1+, older versions use `listen 443 ssl http2;`).
- Never ask for or repeat private keys, passwords or htpasswd hashes.
- Treat security headers like CSP as a rollout: suggest `Content-Security-Policy-Report-Only` first for complex apps.
FILE:references/hardening-checklist.md
# Nginx Hardening Checklist

Every code the script prints, why it matters and the fix. Severity is the default; raise or lower it with context.

## HIGH

### OLD_TLS_PROTOCOL
`ssl_protocols` lists SSLv3, TLSv1 or TLSv1.1. These are deprecated (RFC 8996), fail PCI DSS and allow downgrade attacks. Fix: `ssl_protocols TLSv1.2 TLSv1.3;` (the default since nginx 1.23.4, so deleting the line also works on new versions). Only very old clients (Android 4.3, IE on Windows 7 without updates) lose access.

### WEAK_CIPHERS
`ssl_ciphers` allows RC4, DES/3DES (SWEET32), MD5, NULL, EXPORT or anonymous (aNULL/ADH) suites. Entries starting with `!` or `-` are exclusions and are fine. Fix: use the Mozilla "intermediate" list from ssl-config.mozilla.org or remove `ssl_ciphers` to use the OpenSSL default, and set `ssl_prefer_server_ciphers off;` when only modern AEAD suites remain.

### NO_CERTIFICATE
A server listens with `ssl` but no `ssl_certificate` is set in it or in http. nginx refuses to start or fails the handshake. Add `ssl_certificate` (full chain) and `ssl_certificate_key`.

### ALIAS_TRAVERSAL (off-by-slash)
`location /images { alias /var/www/media/images/; }`: a request for `/images../secret.txt` maps to `/var/www/media/images/../secret.txt`, so files one level above the alias are readable (a well-known finding in many audits). Fix: both the location and the alias end with a slash, `location /images/ { alias /var/www/media/images/; }`, or use `root` when the path matches the folder name.

## MEDIUM

### HTTP_NO_REDIRECT
A port-80 server with the same server_name as an HTTPS server serves pages instead of redirecting. Users and cookies stay on plain HTTP. Fix:
```
server {
    listen 80;
    server_name shop.example.com;
    location /.well-known/acme-challenge/ { root /var/www/certbot; }
    location / { return 301 https://$host$request_uri; }
}
```

### NO_HSTS
No `Strict-Transport-Security` header on an HTTPS server, so the first request of a session can be downgraded. See `tls-and-headers.md` for the rollout.

### ADD_HEADER_INHERITANCE
nginx inherits `add_header` from the parent level only when the current level has no `add_header` at all. One `add_header Cache-Control no-store;` in `location /api/` drops every security header from the server block for that location. Fix: repeat the headers in that location, or keep them in a snippet (`include snippets/security-headers.conf;`) and include it in every block that has its own add_header. On nginx 1.29.3+ `add_header_inherit merge;` in http makes child blocks add to the parent headers instead of replacing them. `proxy_set_header`, `fastcgi_param` and `proxy_hide_header` follow the same rule.

### AUTOINDEX_ON
Directory listing shows every file name, including backups and exports nobody linked. Turn it off or protect the location.

### ADMIN_OPEN
Paths such as /admin, /phpmyadmin, /actuator, /server-status, /nginx_status or /metrics without allow/deny, auth_basic or auth_request. Fix: `allow 10.20.0.0/16; deny all;` (office or VPN range), or `auth_request` to an SSO service. Basic auth only over HTTPS.

### UNLIMITED_BODY
`client_max_body_size 0` disables the request size limit, making disk and memory exhaustion easy. Keep the default 1m or a real limit in http, and raise it only in the upload location.

### URI_IN_REDIRECT
`$uri` is normalized and URL-decoded, so `%0d%0a` becomes a real line break and an attacker can inject headers (`Set-Cookie`) into the redirect response. Use `$request_uri` (the raw, still-encoded request line) in `return`, `rewrite` targets and `add_header`.

## LOW

### SERVER_TOKENS
The version appears in the `Server` header and on error pages, which helps attackers match known CVEs. `server_tokens off;` in http.

### MISSING_SECURITY_HEADERS
On HTTPS servers: `X-Content-Type-Options nosniff`, `X-Frame-Options` (or CSP `frame-ancestors`), `Content-Security-Policy`, `Referrer-Policy`. Add them with `always` so they are also sent on 4xx/5xx responses.

### SHORT_HSTS
HSTS max-age under 180 days. Fine during rollout; raise it to one year afterwards.

### DOTFILES_EXPOSED
The server serves files but has no rule for hidden files, so `/.git/config`, `/.env` or `/.htpasswd` may be downloadable if they exist under the web root. Fix: `location ~ /\.(?!well-known) { deny all; }` (keeps ACME challenges working).

### NO_RATE_LIMIT
A login, auth or token path without `limit_req`. Add a zone in http and apply it to the location; return 429 with `limit_req_status 429;`.

### PROXY_HEADERS
A `proxy_pass` location that does not send Host, X-Forwarded-For and X-Forwarded-Proto, so the app logs the proxy's IP, builds wrong redirect URLs and may think HTTPS requests are HTTP. Remember the inheritance rule: one `proxy_set_header` in the location replaces all inherited ones.

## INFO

### INCLUDE_NOT_FOLLOWED
The script reads only the files you pass. Ask for `nginx -T` or the included files.

## Check by hand (not automated)
- `proxy_pass https://...` to an upstream on the internet without `proxy_ssl_verify on;` and `proxy_ssl_trusted_certificate`.
- PHP: `location ~ \.php$` must use `try_files $uri =404;` (or `fastcgi_split_path_info` with a check) so uploaded images cannot run as PHP; uploads directories should never pass to PHP.
- `if` inside location with anything other than `return` or `rewrite ... last` ("if is evil").
- `default_server` blocks: an unknown Host should get `return 444;` with a self-signed or `ssl_reject_handshake on;` (1.19.4+) instead of the first site.
- Permissions: nginx workers should not be able to write the web root; private keys `chmod 600`, owned by root.
- Large headers / slow clients: `client_body_timeout`, `client_header_timeout`, `large_client_header_buffers` for public APIs under attack.
- Logging: access logs must not record tokens in query strings; use a custom log_format if needed.
FILE:references/tls-and-headers.md
# TLS and Security Headers

## TLS settings that pass most audits (nginx 1.25+ with OpenSSL 1.1.1 or 3.x)
```
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
```
This is the Mozilla "intermediate" profile. TLS 1.3 suites are not set with ssl_ciphers (OpenSSL handles them). If DHE suites stay, provide `ssl_dhparam` with at least 2048 bits or drop the DHE entries.

OCSP stapling (`ssl_stapling on; ssl_stapling_verify on;`) is optional; Let's Encrypt ended OCSP support in 2025, so do not add it for LE certificates.

HTTP/2: `http2 on;` inside the server on nginx 1.25.1+; older versions use `listen 443 ssl http2;`.

## HSTS rollout
1. Make sure every subdomain works over HTTPS.
2. Start short: `add_header Strict-Transport-Security "max-age=300" always;`
3. After a week without problems: `max-age=31536000; includeSubDomains`.
4. Only add `preload` and submit to hstspreload.org when you are sure; removal takes months.
HSTS on plain HTTP responses is ignored by browsers, so set it on the 443 server.

## Security headers
| Header | Typical value | Notes |
|---|---|---|
| X-Content-Type-Options | `nosniff` | Stops MIME sniffing of uploads |
| X-Frame-Options | `DENY` or `SAMEORIGIN` | Clickjacking; CSP `frame-ancestors` is the modern equivalent |
| Content-Security-Policy | `default-src 'self'` and more | App-specific; roll out with `Content-Security-Policy-Report-Only` |
| Referrer-Policy | `strict-origin-when-cross-origin` | Keeps paths and query strings out of other sites' logs |
| Permissions-Policy | `camera=(), microphone=(), geolocation=()` | Optional |
Add `always` so headers are also sent on error responses.

## The add_header inheritance rule
```
server {
    add_header X-Content-Type-Options nosniff always;   # A
    location / { }                                      # gets A
    location /api/ {
        add_header Cache-Control no-store;              # gets ONLY this one, A is gone
    }
}
```
A block inherits add_header from its parent only if it defines none itself. Keep shared headers in a snippet and include it in every block that adds its own. nginx 1.29.3 (October 2025) added `add_header_inherit merge;`, which makes nested blocks add to the inherited headers; check `nginx -v` before suggesting it (Debian 13 ships 1.26).

## Testing
```
sudo nginx -t && sudo systemctl reload nginx
curl -sI https://shop.example.com/ | grep -iE 'strict|x-content|x-frame|content-security|referrer|server:'
curl -sI http://shop.example.com/page?x=1        # expect 301 to https://shop.example.com/page?x=1
curl -s -o /dev/null -w '%{http_code}\n' https://shop.example.com/.git/config   # expect 403 or 404
curl -s -o /dev/null -w '%{http_code}\n' https://shop.example.com/images../      # expect 404 after the alias fix
openssl s_client -connect shop.example.com:443 -tls1_1 </dev/null   # expect a handshake failure
```
For a full external check: SSL Labs server test or `testssl.sh shop.example.com`.
FILE:templates/review-report.md
# Nginx security review: <site or server name>

**Version:** <nginx 1.26.3, OpenSSL 3.x>   **Role:** <static site / reverse proxy to Node on :3000 / PHP-FPM>
**Files reviewed:** <nginx -T output, 2 files>   **TLS terminated by:** <nginx / CDN / load balancer>
**Summary:** <one sentence: the most serious problem and what the fix changes>

## Findings
| # | Severity | Code | File:line | Problem | Fix |
|---|---|---|---|---|---|
| 1 | HIGH | <ALIAS_TRAVERSAL> | <nginx.conf:51> | <location /images allows /images../> | <location /images/ { alias .../images/; }> |
| 2 | <MEDIUM> | <...> | <...> | <...> | <...> |

## Not changed (with reason)
- <ADMIN_OPEN on /admin/: already limited by the cloud firewall to the VPN; added deny all anyway? yes/no>

## Config changes
```diff
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
+ ssl_protocols TLSv1.2 TLSv1.3;
```

## Apply and test
1. Back up: `sudo cp -a /etc/nginx /etc/nginx.bak-<date>`
2. Edit, then `sudo nginx -t && sudo systemctl reload nginx`
3. Checks:
   - `curl -sI https://<host>/` shows the security headers
   - `curl -sI http://<host>/x?y=1` returns 301 to the same path on https
   - <path traversal / dotfile / admin checks from the findings>
4. Roll back: restore the backup and reload.

## Follow-ups
- <HSTS: raise max-age to 31536000 after <date>>
- <CSP: move from Report-Only to enforcing after reviewing reports>
FILE:examples/shop-site.conf
# /etc/nginx/nginx.conf for shop.example.com (single-file setup)
# A small online shop: static storefront, product images, a Node.js API on :3000
# and a legacy admin panel. Written in 2019, certificates renewed by certbot.

user www-data;
worker_processes auto;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    sendfile on;
    keepalive_timeout 65;
    client_max_body_size 0;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "HIGH:RC4-SHA:DES-CBC3-SHA:!aNULL";
    ssl_prefer_server_ciphers on;

    server {
        listen 80;
        server_name shop.example.com www.shop.example.com;
        root /var/www/shop;

        location /.well-known/acme-challenge/ {
            root /var/www/certbot;
        }
    }

    server {
        listen 443 ssl;
        http2 on;
        server_name shop.example.com www.shop.example.com;

        ssl_certificate /etc/letsencrypt/live/shop.example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/shop.example.com/privkey.pem;

        add_header Strict-Transport-Security "max-age=31536000" always;
        add_header X-Content-Type-Options nosniff always;

        root /var/www/shop;
        index index.html;

        location / {
            try_files $uri $uri/ /index.html;
        }

        location /images {
            alias /var/www/shop-media/images/;
        }

        location /downloads/ {
            alias /var/www/shop-media/downloads/;
            autoindex on;
        }

        location /api/ {
            add_header Cache-Control "no-store";
            proxy_pass http://127.0.0.1:3000;
        }

        location /api/auth/login {
            proxy_pass http://127.0.0.1:3000;
            proxy_set_header Host $host;
        }

        location /admin/ {
            proxy_pass http://127.0.0.1:8080/;
        }

        location /old-blog {
            return 301 https://blog.example.com$uri;
        }
    }
}
FILE:examples/shop-site-fixed.conf
# shop.example.com after the review: same routes, hardened.
# Shared security headers live in one snippet so locations with their own
# add_header can include it again (add_header is not inherited into a block
# that sets any add_header of its own).

user www-data;
worker_processes auto;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    sendfile on;
    keepalive_timeout 65;
    server_tokens off;
    client_max_body_size 10m;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;

    limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;

    server {
        listen 80;
        server_name shop.example.com www.shop.example.com;

        location /.well-known/acme-challenge/ {
            root /var/www/certbot;
        }

        location / {
            return 301 https://$host$request_uri;
        }
    }

    server {
        listen 443 ssl;
        http2 on;
        server_name shop.example.com www.shop.example.com;

        ssl_certificate /etc/letsencrypt/live/shop.example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/shop.example.com/privkey.pem;

        # same lines as /etc/nginx/snippets/security-headers.conf
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header X-Content-Type-Options nosniff always;
        add_header X-Frame-Options SAMEORIGIN always;
        add_header Content-Security-Policy "default-src 'self'; img-src 'self' data:; frame-ancestors 'self'" always;
        add_header Referrer-Policy strict-origin-when-cross-origin always;

        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        root /var/www/shop;
        index index.html;

        location ~ /\.(?!well-known) {
            deny all;
        }

        location / {
            try_files $uri $uri/ /index.html;
        }

        location /images/ {
            alias /var/www/shop-media/images/;
        }

        location /downloads/ {
            alias /var/www/shop-media/downloads/;
        }

        location /api/ {
            add_header Cache-Control "no-store" always;
            add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
            add_header X-Content-Type-Options nosniff always;
            add_header X-Frame-Options SAMEORIGIN always;
            add_header Content-Security-Policy "default-src 'self'; img-src 'self' data:; frame-ancestors 'self'" always;
            add_header Referrer-Policy strict-origin-when-cross-origin always;
            proxy_pass http://127.0.0.1:3000;
        }

        location /api/auth/login {
            limit_req zone=login burst=5 nodelay;
            proxy_pass http://127.0.0.1:3000;
        }

        location /admin/ {
            allow 10.20.0.0/16;
            deny all;
            proxy_pass http://127.0.0.1:8080/;
        }

        location /old-blog {
            return 301 https://blog.example.com$request_uri;
        }
    }
}
FILE:scripts/nginx_review.py
#!/usr/bin/env python3
"""Nginx config security reviewer (Python 3 standard library only).

Parses nginx.conf / sites-available files into a block tree and flags common
security and hardening problems: old TLS protocols, weak ciphers, missing HSTS
and security headers, the add_header inheritance trap, alias off-by-slash path
traversal, autoindex, unlimited body size, $uri in redirects (CRLF injection),
plain-HTTP servers without a redirect, open admin paths, login paths without
rate limiting, exposed dotfiles and proxies without forwarding headers.

Usage:
  python3 nginx_review.py site.conf [more.conf ...] [--json]
  python3 nginx_review.py --self-test
Exit codes: 0 = no HIGH findings, 1 = HIGH findings, 2 = unreadable/unparseable input.
"""
import argparse, json, re, sys

SEV_ORDER = {"HIGH": 0, "MEDIUM": 1, "LOW": 2, "INFO": 3}
OLD_PROTOCOLS = {"SSLv2", "SSLv3", "TLSv1", "TLSv1.1"}
WEAK_CIPHER_WORDS = ["RC4", "DES", "3DES", "MD5", "NULL", "EXPORT", "EXP-", "aNULL", "eNULL", "ADH", "AECDH"]
SECURITY_HEADERS = {
    "x-content-type-options": "X-Content-Type-Options nosniff",
    "x-frame-options": "X-Frame-Options DENY or SAMEORIGIN (or CSP frame-ancestors)",
    "content-security-policy": "Content-Security-Policy",
    "referrer-policy": "Referrer-Policy strict-origin-when-cross-origin",
}
LOGIN_RE = re.compile(r"login|signin|sign-in|auth|token|password|wp-login", re.I)
ADMIN_RE = re.compile(r"admin|phpmyadmin|actuator|server-status|nginx_status|/metrics|wp-admin", re.I)


class ParseError(Exception):
    pass


class Node:
    def __init__(self, name, args, line, children=None):
        self.name, self.args, self.line, self.children = name, args, line, children

    def get(self, name):
        return [c for c in (self.children or []) if c.name == name]

    def first(self, name):
        found = self.get(name)
        return found[0] if found else None

    def walk(self):
        for c in self.children or []:
            yield c
            if c.children is not None:
                yield from c.walk()


def tokenize(text):
    tokens, i, line, n = [], 0, 1, len(text)
    while i < n:
        ch = text[i]
        if ch == "\n":
            line += 1; i += 1
        elif ch.isspace():
            i += 1
        elif ch == "#":
            while i < n and text[i] != "\n":
                i += 1
        elif ch in "{};":
            tokens.append((ch, line, True)); i += 1
        elif ch in "\"'":
            q, j, buf = ch, i + 1, []
            while j < n and text[j] != q:
                if text[j] == "\\" and j + 1 < n:
                    buf.append(text[j + 1]); j += 2; continue
                if text[j] == "\n":
                    line += 1
                buf.append(text[j]); j += 1
            if j >= n:
                raise ParseError(f"line {line}: unterminated quote")
            tokens.append(("".join(buf), line, False)); i = j + 1
        else:
            j = i
            while j < n and not text[j].isspace() and text[j] not in "{};":
                if text[j] == "$" and j + 1 < n and text[j + 1] == "{":  # var
                    k = text.find("}", j)
                    j = k + 1 if k != -1 else n
                    continue
                j += 1
            tokens.append((text[i:j], line, False)); i = j
    return tokens


def parse(text):
    tokens = tokenize(text)
    root = Node("main", [], 0, [])
    stack, cur = [root], []
    for tok, line, special in tokens:
        if special and tok == ";":
            if not cur:
                continue
            stack[-1].children.append(Node(cur[0][0], [t for t, _ in cur[1:]], cur[0][1]))
            cur = []
        elif special and tok == "{":
            if not cur:
                raise ParseError(f"line {line}: '{{' without a directive name")
            node = Node(cur[0][0], [t for t, _ in cur[1:]], cur[0][1], [])
            stack[-1].children.append(node); stack.append(node); cur = []
        elif special and tok == "}":
            if cur:
                raise ParseError(f"line {line}: missing ';' before '}}'")
            if len(stack) == 1:
                raise ParseError(f"line {line}: unexpected '}}'")
            stack.pop()
        else:
            cur.append((tok, line))
    if cur:
        raise ParseError(f"line {cur[0][1]}: directive '{cur[0][0]}' not terminated with ';'")
    if len(stack) != 1:
        raise ParseError(f"block '{stack[-1].name}' opened on line {stack[-1].line} is not closed")
    return root


def headers_of(node):
    return {d.args[0].lower(): d for d in node.get("add_header") if d.args}


def is_tls(server):
    for l in server.get("listen"):
        if "ssl" in l.args or "quic" in l.args or any(a.endswith(":443") or a == "443" for a in l.args):
            return True
    s = server.first("ssl")
    return bool(s and s.args and s.args[0] == "on")


def listens_plain(server):
    ls = server.get("listen")
    if not ls:
        return True  # default listen *:80
    for l in ls:
        if not ("ssl" in l.args or "quic" in l.args or any(a.endswith(":443") or a == "443" for a in l.args)):
            return True
    return False


def names(server):
    out = []
    for d in server.get("server_name"):
        out.extend(d.args)
    return set(out) or {"_"}


def inherited(scopes, name):
    """Value of a directive as nginx inherits it: innermost scope that sets it."""
    for s in reversed(scopes):
        d = s.first(name)
        if d is not None:
            return d
    return None


def effective_headers(scopes):
    for s in reversed(scopes):
        h = headers_of(s)
        if h:
            return h
    return {}


def review(text, fname="nginx.conf"):
    root = parse(text)
    http = root.first("http")
    top = http if http is not None else root
    base = [root] + ([http] if http is not None else [])
    servers = [s for s in top.get("server")]
    findings = []

    def add(sev, code, line, msg, fix):
        findings.append({"severity": sev, "code": code, "file": fname, "line": line, "message": msg, "fix": fix})

    for d in root.walk():
        if d.name == "include":
            add("INFO", "INCLUDE_NOT_FOLLOWED", d.line, f"include {' '.join(d.args)} is not read by this checker",
                "Review the included files too (pass them as extra arguments) or paste `nginx -T` output.")

    tokens = [d for d in root.walk() if d.name == "server_tokens"]
    if not any(d.args and d.args[0] == "off" for d in tokens):
        add("LOW", "SERVER_TOKENS", (http or root).line, "server_tokens is not off, so error pages and the Server header show the nginx version",
            "Add `server_tokens off;` in the http block.")

    tls_names = set()
    for s in servers:
        if is_tls(s):
            tls_names |= names(s)

    for s in servers:
        scopes = base + [s]
        label = " ".join(sorted(names(s)))
        tls = is_tls(s)
        if tls:
            if inherited(scopes, "ssl_certificate") is None:
                add("HIGH", "NO_CERTIFICATE", s.line, f"TLS server {label} has no ssl_certificate in scope",
                    "Add ssl_certificate and ssl_certificate_key (fullchain + key).")
            proto = inherited(scopes, "ssl_protocols")
            if proto is not None:
                old = [p for p in proto.args if p in OLD_PROTOCOLS]
                if old:
                    add("HIGH", "OLD_TLS_PROTOCOL", proto.line, f"{label}: ssl_protocols enables {', '.join(old)}",
                        "Use `ssl_protocols TLSv1.2 TLSv1.3;` (SSLv3/TLS 1.0/1.1 are deprecated, RFC 8996).")
            ciph = inherited(scopes, "ssl_ciphers")
            if ciph is not None and ciph.args:
                parts = re.split(r"[:\s]+", " ".join(ciph.args))
                weak = sorted({w for p in parts if p and not p.startswith("!") and not p.startswith("-")
                               for w in WEAK_CIPHER_WORDS if w.lower() in p.lower()})
                if weak:
                    add("HIGH", "WEAK_CIPHERS", ciph.line, f"{label}: ssl_ciphers allows weak suites ({', '.join(weak)})",
                        "Use the Mozilla 'intermediate' cipher list, or remove ssl_ciphers to use the safe OpenSSL default.")
            hdrs = effective_headers(scopes)
            if "strict-transport-security" not in hdrs:
                add("MEDIUM", "NO_HSTS", s.line, f"TLS server {label} sends no Strict-Transport-Security header",
                    'Add `add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;` (start with a short max-age).')
            else:
                hsts = hdrs["strict-transport-security"]
                m = re.search(r"max-age=(\d+)", " ".join(hsts.args), re.I)
                if m and int(m.group(1)) < 15552000:
                    add("LOW", "SHORT_HSTS", hsts.line, f"{label}: HSTS max-age {m.group(1)} is under 180 days",
                        "Raise max-age to 31536000 once HTTPS works everywhere.")
            missing = [v for k, v in SECURITY_HEADERS.items() if k not in hdrs]
            if "x-frame-options" not in hdrs and "content-security-policy" in hdrs and \
                    "frame-ancestors" in " ".join(hdrs["content-security-policy"].args):
                missing = [v for v in missing if not v.startswith("X-Frame")]
            if missing:
                add("LOW", "MISSING_SECURITY_HEADERS", s.line, f"{label}: missing {'; '.join(missing)}",
                    "Add them with `always` in the server block (check the inheritance rule for locations).")
        elif listens_plain(s) and names(s) & tls_names:
            redirects = any(r.args and r.args[0] in ("301", "302", "307", "308") and len(r.args) > 1 and r.args[1].startswith("https://")
                            for r in [d for d in s.walk() if d.name == "return"])
            rewrites = any(len(r.args) > 1 and r.args[1].startswith("https://") for r in [d for d in s.walk() if d.name == "rewrite"])
            if not (redirects or rewrites):
                add("MEDIUM", "HTTP_NO_REDIRECT", s.line, f"plain-HTTP server {label} serves content instead of redirecting to HTTPS",
                    "Use `return 301 https://$host$request_uri;` in the port-80 server (keep only /.well-known/acme-challenge/ if needed).")

        server_hdrs = effective_headers(scopes)
        has_dot_rule = False
        locations = [d for d in s.walk() if d.name == "location"]
        for loc in locations:
            path = loc.args[-1] if loc.args else ""
            mod = loc.args[0] if len(loc.args) > 1 else ""
            if mod in ("~", "~*") and (r"/\." in path or path.startswith(r"\.") or r"(^|/)\." in path):
                if loc.get("deny") or any(r.args and r.args[0] in ("403", "404") for r in loc.get("return")):
                    has_dot_rule = True
            own = headers_of(loc)
            hi = inherited(scopes + [loc], "add_header_inherit")
            merge = bool(hi and hi.args and hi.args[0] == "merge")  # nginx 1.29.3+
            if own and server_hdrs and not merge:
                lost = sorted(set(server_hdrs) - set(own))
                if lost:
                    add("MEDIUM", "ADD_HEADER_INHERITANCE", loc.line,
                        f"location {path} has its own add_header, so it silently drops {', '.join(lost)} set at server/http level",
                        "Repeat those headers in this location, include a shared headers snippet, or on nginx 1.29.3+ set `add_header_inherit merge;`.")
            al = loc.first("alias")
            if al and al.args and mod in ("", "^~") and not path.endswith("/") and al.args[0].endswith("/"):
                add("HIGH", "ALIAS_TRAVERSAL", al.line,
                    f"location {path} (no trailing slash) with alias {al.args[0]} (trailing slash) allows {path}../ path traversal",
                    f"Make both end with a slash: `location {path}/ {{ alias {al.args[0]}; }}`.")
            ai = loc.first("autoindex")
            if ai and ai.args and ai.args[0] == "on":
                add("MEDIUM", "AUTOINDEX_ON", ai.line, f"autoindex on in location {path} lists directory contents",
                    "Turn it off, or restrict the location with allow/deny or auth_basic.")
            if ADMIN_RE.search(path):
                guarded = loc.get("deny") or loc.get("auth_basic") or loc.get("auth_request") or loc.get("satisfy") \
                    or any(r.args and r.args[0] in ("403", "404", "444") for r in loc.get("return"))
                if not guarded:
                    add("MEDIUM", "ADMIN_OPEN", loc.line, f"location {path} looks like an admin/status endpoint with no allow/deny or auth",
                        "Add `allow <office/VPN range>; deny all;` or auth_basic/auth_request, or remove the route.")
            if LOGIN_RE.search(path) and not inherited(scopes + [loc], "limit_req"):
                add("LOW", "NO_RATE_LIMIT", loc.line, f"location {path} looks like a login/auth endpoint without limit_req",
                    "Define `limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;` in http and `limit_req zone=login burst=5 nodelay;` here.")
            if loc.first("proxy_pass"):
                ph = set()
                for sc in reversed(scopes + [loc]):  # like add_header: innermost level that sets any wins
                    if sc.get("proxy_set_header"):
                        ph = {d.args[0].lower() for d in sc.get("proxy_set_header") if d.args}
                        break
                miss = [h for h in ("host", "x-forwarded-for", "x-forwarded-proto") if h not in ph]
                if miss:
                    add("LOW", "PROXY_HEADERS", loc.line, f"location {path} proxies without {', '.join(miss)}",
                        "Set proxy_set_header Host $host; X-Forwarded-For $proxy_add_x_forwarded_for; X-Forwarded-Proto $scheme; (note: any proxy_set_header in a location replaces inherited ones).")
        serves_files = inherited(scopes, "root") or any(
            (l.first("root") or l.first("alias")) and not (l.args and l.args[-1].startswith("/.well-known/")) for l in locations)
        if serves_files and not has_dot_rule:
            add("LOW", "DOTFILES_EXPOSED", s.line, f"{label}: no rule blocks hidden files such as /.git/ or /.env",
                r"Add `location ~ /\.(?!well-known) { deny all; }`.")

    for d in root.walk():
        if d.name == "client_max_body_size" and d.args and d.args[0] in ("0", "0m", "0k"):
            add("MEDIUM", "UNLIMITED_BODY", d.line, "client_max_body_size 0 disables the request body limit",
                "Set a real limit (e.g. 10m, or larger only in the upload location).")
        if d.name in ("return", "rewrite", "add_header") and any(re.search(r"\$uri\b|\$document_uri\b", a) for a in d.args):
            add("MEDIUM", "URI_IN_REDIRECT", d.line, f"{d.name} uses $uri (decoded, may contain CR/LF), which allows header injection",
                "Use $request_uri (raw, still encoded) in redirects and headers.")
    findings.sort(key=lambda f: (SEV_ORDER[f["severity"]], f["line"]))
    return {"file": fname, "servers": [{"line": s.line, "names": sorted(names(s)), "tls": is_tls(s)} for s in servers],
            "findings": findings}


def to_markdown(results):
    out = ["# Nginx config security review", ""]
    allf = [f for r in results for f in r["findings"]]
    counts = {k: sum(1 for f in allf if f["severity"] == k) for k in SEV_ORDER}
    out.append("Findings: " + ", ".join(f"{v} {k}" for k, v in counts.items()))
    for r in results:
        out += ["", f"## {r['file']}", "", "| Server | Line | TLS |", "|---|---|---|"]
        for s in r["servers"]:
            out.append(f"| {' '.join(s['names'])} | {s['line']} | {'yes' if s['tls'] else 'no'} |")
        out += ["", "| Severity | Code | Line | Problem | Fix |", "|---|---|---|---|---|"]
        for f in r["findings"]:
            out.append(f"| {f['severity']} | {f['code']} | {f['line']} | {f['message']} | {f['fix']} |")
        if not r["findings"]:
            out.append("| - | - | - | no findings | - |")
    out += ["", "Always run `nginx -t` before reloading, and test with `curl -sI https://<host>/` afterwards."]
    return "\n".join(out)


def self_test():
    results = []

    def check(name, cond):
        results.append(cond)
        print(("PASS " if cond else "FAIL ") + name)

    def codes(conf):
        return [f["code"] for f in review(conf)["findings"]]

    good = """
    http {
      server_tokens off;
      limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;
      server { listen 80; server_name a.test; return 301 https://$host$request_uri; }
      server {
        listen 443 ssl; http2 on; server_name a.test;
        ssl_certificate /c.pem; ssl_certificate_key /k.pem;
        ssl_protocols TLSv1.2 TLSv1.3;
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header X-Content-Type-Options nosniff always;
        add_header Content-Security-Policy "default-src 'self'; frame-ancestors 'none'" always;
        add_header Referrer-Policy strict-origin-when-cross-origin always;
        root /srv/www;
        location ~ /\\.(?!well-known) { deny all; }
        location /static/ { alias /srv/static/; }
        location /login { limit_req zone=login burst=5 nodelay; proxy_pass http://app;
          proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Forwarded-Proto $scheme; }
      }
    }"""
    check("clean config has no findings", codes(good) == [])
    check("comments and quotes parsed", parse('a "b c;" d; # x;\n e f;').children[0].args == ["b c;", "d"])
    check("var token kept", parse("set $x hosta;").children[0].args == ["$x", "hosta"])
    for bad, msg in [("server { listen 80;", "unclosed block"), ("a b", "missing ;"), ("}", "stray brace"), ('a "b;', "unterminated quote")]:
        try:
            parse(bad); check("parse error: " + msg, False)
        except ParseError:
            check("parse error: " + msg, True)
    check("SERVER_TOKENS when missing", "SERVER_TOKENS" in codes("http { server { listen 80; } }"))
    tls = "server { listen 443 ssl; server_name b.test; ssl_certificate /c; %s }"
    check("OLD_TLS_PROTOCOL", "OLD_TLS_PROTOCOL" in codes(tls % "ssl_protocols TLSv1 TLSv1.2;"))
    check("TLSv1.2 alone fine", "OLD_TLS_PROTOCOL" not in codes(tls % "ssl_protocols TLSv1.2;"))
    check("WEAK_CIPHERS", "WEAK_CIPHERS" in codes(tls % "ssl_ciphers 'HIGH:RC4-SHA:DES-CBC3-SHA';"))
    check("excluded !RC4 is fine", "WEAK_CIPHERS" not in codes(tls % "ssl_ciphers 'HIGH:!aNULL:!MD5:!RC4';"))
    check("NO_CERTIFICATE", "NO_CERTIFICATE" in codes("server { listen 443 ssl; }"))
    check("cert inherited from http", "NO_CERTIFICATE" not in codes("http { ssl_certificate /c; server { listen 443 ssl; } }"))
    check("NO_HSTS", "NO_HSTS" in codes(tls % ""))
    check("SHORT_HSTS", "SHORT_HSTS" in codes(tls % 'add_header Strict-Transport-Security "max-age=300";'))
    check("MISSING_SECURITY_HEADERS", "MISSING_SECURITY_HEADERS" in codes(tls % ""))
    inh = tls % 'add_header Strict-Transport-Security "max-age=31536000"; location /api/ { add_header Cache-Control no-store; }'
    r = review(inh)["findings"]
    check("ADD_HEADER_INHERITANCE names lost header",
          any(f["code"] == "ADD_HEADER_INHERITANCE" and "strict-transport-security" in f["message"] for f in r))
    check("headers inherited when location has none", "ADD_HEADER_INHERITANCE" not in codes(
        tls % 'add_header Strict-Transport-Security "max-age=31536000"; location /api/ { proxy_pass http://x; }'))
    check("add_header_inherit merge respected", "ADD_HEADER_INHERITANCE" not in codes(
        tls % 'add_header_inherit merge; add_header Strict-Transport-Security "max-age=31536000"; location /api/ { add_header Cache-Control no-store; }'))
    check("ALIAS_TRAVERSAL", "ALIAS_TRAVERSAL" in codes("server { location /img { alias /srv/img/; } }"))
    check("alias with matching slashes fine", "ALIAS_TRAVERSAL" not in codes("server { location /img/ { alias /srv/img/; } }"))
    check("regex location alias not flagged", "ALIAS_TRAVERSAL" not in codes(r"server { location ~ ^/img/(.*)$ { alias /srv/img/$1; } }"))
    check("AUTOINDEX_ON", "AUTOINDEX_ON" in codes("server { location /files/ { autoindex on; } }"))
    check("HTTP_NO_REDIRECT", "HTTP_NO_REDIRECT" in codes(
        "server { listen 80; server_name c.test; root /x; } server { listen 443 ssl; server_name c.test; ssl_certificate /c; }"))
    check("no HTTP_NO_REDIRECT for unrelated name", "HTTP_NO_REDIRECT" not in codes(
        "server { listen 80; server_name d.test; } server { listen 443 ssl; server_name c.test; ssl_certificate /c; }"))
    check("ADMIN_OPEN", "ADMIN_OPEN" in codes("server { location /admin/ { proxy_pass http://x; } }"))
    check("admin with deny fine", "ADMIN_OPEN" not in codes("server { location /admin/ { allow 10.0.0.0/8; deny all; } }"))
    check("NO_RATE_LIMIT", "NO_RATE_LIMIT" in codes("server { location /api/login { proxy_pass http://x; } }"))
    check("rate limit inherited from server", "NO_RATE_LIMIT" not in codes(
        "server { limit_req zone=z; location /api/login { proxy_pass http://x; } }"))
    check("PROXY_HEADERS", "PROXY_HEADERS" in codes("server { location / { proxy_pass http://x; } }"))
    check("proxy_set_header in location replaces inherited", "PROXY_HEADERS" in codes(
        "server { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; "
        "proxy_set_header X-Forwarded-Proto $scheme; location / { proxy_set_header Upgrade $http_upgrade; proxy_pass http://x; } }"))
    check("DOTFILES_EXPOSED", "DOTFILES_EXPOSED" in codes("server { root /srv; }"))
    check("ACME-only port 80 server not flagged for dotfiles", "DOTFILES_EXPOSED" not in codes(
        "server { listen 80; location /.well-known/acme-challenge/ { root /var/www/certbot; } location / { return 301 https://$host$request_uri; } }"))
    check("UNLIMITED_BODY", "UNLIMITED_BODY" in codes("http { client_max_body_size 0; }"))
    check("URI_IN_REDIRECT", "URI_IN_REDIRECT" in codes("server { return 301 https://$host$uri; }"))
    check("$request_uri fine", "URI_IN_REDIRECT" not in codes("server { return 301 https://$host$request_uri; }"))
    check("INCLUDE_NOT_FOLLOWED", "INCLUDE_NOT_FOLLOWED" in codes("http { include /etc/nginx/conf.d/*.conf; }"))
    check("findings sorted HIGH first", review("server { listen 443 ssl; location /files/ { autoindex on; } }")["findings"][0]["severity"] == "HIGH")
    passed = sum(results)
    print(f"{passed}/{len(results)} passed")
    return 0 if passed == len(results) else 1


def main():
    ap = argparse.ArgumentParser(description="Nginx config security reviewer")
    ap.add_argument("files", nargs="*", help="nginx config files (nginx.conf, sites-available/*, or `nginx -T` output)")
    ap.add_argument("--json", action="store_true", help="print JSON instead of Markdown")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args()
    if a.self_test:
        return self_test()
    if not a.files:
        ap.error("give at least one config file, or --self-test")
    results = []
    for f in a.files:
        try:
            with open(f, encoding="utf-8") as fh:
                results.append(review(fh.read(), f))
        except (OSError, UnicodeDecodeError, ParseError) as e:
            print(f"error: {f}: {e}", file=sys.stderr)
            return 2
    print(json.dumps(results, indent=2) if a.json else to_markdown(results))
    return 1 if any(f["severity"] == "HIGH" for r in results for f in r["findings"]) else 0


if __name__ == "__main__":
    sys.exit(main())