Best Practices Prompts

# Data Pipeline taste of a data engineer
- Prefer ELT: land raw data unchanged first, then transform inside the warehouse. Confidence: 0.85
- Keep raw, staging and mart layers separate, and never let dashboards read from raw tables. Confidence: 0.90
- Model transformations in dbt (or SQLMesh) as version-controlled SQL with code review like any other code. Confidence: 0.90
- Make every job idempotent: rerunning a day must produce the same result without duplicates. Confidence: 0.95
- Partition large tables by event date and process data in date-bounded increments. Confidence: 0.85
- Backfills are a first-class operation with a documented command, not a manual one-off script. Confidence: 0.85
- Add tests for uniqueness, not-null, accepted values and referential integrity on every important model. Confidence: 0.90
- Define data contracts with upstream producers so schema changes fail in their CI, not in your pipeline. Confidence: 0.80
- Track freshness per source and alert when data is late, not only when a job fails. Confidence: 0.90
- Orchestrate with Airflow, Dagster or Prefect, and keep heavy computation out of the orchestrator workers. Confidence: 0.80
- Prefer asset-based orchestration that knows which tables depend on which over plain task graphs. Confidence: 0.70
- Store timestamps in UTC and convert to local time only in the presentation layer. Confidence: 0.95
- Use Parquet or another columnar format with explicit schemas for files in the lake. Confidence: 0.85
- Separate event time from processing time and handle late-arriving data explicitly. Confidence: 0.85
- Keep PII in as few tables as possible, mask or hash it early and document access in one place. Confidence: 0.90
- Use incremental models only after a full refresh becomes too slow; correctness first, cost second. Confidence: 0.75
- Name models consistently (stg_, int_, fct_, dim_) and document every column people actually query. Confidence: 0.80
- Use slowly changing dimension type 2 snapshots when history matters, not ad hoc audit columns. Confidence: 0.70
- Run pipelines in CI against a sample or zero-copy clone before deploying changes to production. Confidence: 0.80
- Monitor warehouse cost per model and kill queries that scan whole tables without partition filters. Confidence: 0.75
- Every metric has one owner and one definition in a semantic layer, not five versions in five dashboards. Confidence: 0.80

# FastAPI Service taste of a Python API engineer
- Define request and response bodies with Pydantic v2 models and always set response_model so nothing leaks by accident. Confidence: 0.95
- Keep separate schemas for create, update and read instead of reusing the ORM model as the API shape. Confidence: 0.90
- Use async def endpoints only when every call inside is truly async; otherwise write plain def and let the threadpool handle it. Confidence: 0.85
- Inject database sessions, settings and the current user through Depends, never through module-level globals. Confidence: 0.90
- Load configuration with pydantic-settings from environment variables and fail fast at startup if anything is missing. Confidence: 0.90
- Split routes into APIRouter modules by domain and keep main.py limited to app creation and wiring. Confidence: 0.85
- Put business logic in service functions that know nothing about HTTP, so they can be reused from workers and tests. Confidence: 0.85
- Use SQLAlchemy 2.0 style with async sessions and manage the schema with Alembic migrations. Confidence: 0.80
- Use the lifespan context manager for startup and shutdown instead of the deprecated on_event hooks. Confidence: 0.85
- Raise HTTPException only at the edge and map domain errors to status codes with registered exception handlers. Confidence: 0.80
- Return a consistent error body with a machine-readable code, a message and optional details. Confidence: 0.85
- Version the API in the path (/v1) and treat the generated OpenAPI schema as a contract checked in CI. Confidence: 0.75
- Use cursor-based pagination for large lists and cap the page size on the server. Confidence: 0.80
- Test with pytest and httpx.AsyncClient, overriding dependencies instead of patching internals. Confidence: 0.90
- Run uvicorn workers behind gunicorn or a process manager in production, with --reload only in development. Confidence: 0.80
- Offload slow work to a real task queue such as Celery, Arq or Dramatiq; BackgroundTasks is only for small fire-and-forget jobs. Confidence: 0.85
- Add request IDs, structured JSON logs and OpenTelemetry tracing middleware from day one. Confidence: 0.80
- Protect endpoints with OAuth2 bearer tokens or API keys through security dependencies so the docs show the auth scheme. Confidence: 0.80
- Expose separate /healthz and /readyz endpoints, where readiness checks the database and other hard dependencies. Confidence: 0.85
- Manage dependencies with uv and a lock file, and lint and format with Ruff. Confidence: 0.75
- Type check with mypy or pyright in strict mode on the service package. Confidence: 0.70

# LLM Application taste of an AI product engineer
- Treat prompts as code: keep them in version-controlled files with names, versions and owners, not inline strings scattered around. Confidence: 0.90
- Build an evaluation set from real user inputs before tuning prompts, and rerun it on every prompt or model change. Confidence: 0.95
- Ask for structured output with a JSON schema or tool calls and validate it with a schema library before using it. Confidence: 0.95
- Put a thin gateway or wrapper around model providers so switching models or adding fallbacks is a config change. Confidence: 0.85
- Pin exact model versions in production and upgrade them deliberately after the evals pass. Confidence: 0.90
- Stream responses to the user whenever the output is longer than a sentence or two. Confidence: 0.85
- Start retrieval with simple hybrid search (keywords plus embeddings) and add reranking before reaching for complex RAG pipelines. Confidence: 0.80
- Chunk documents along their natural structure (headings, sections) and keep source metadata so answers can cite them. Confidence: 0.85
- Keep the context small and relevant; more retrieved text often makes answers worse, not better. Confidence: 0.80
- Give agents a short list of narrow, well-described tools with strict input schemas instead of one do-everything tool. Confidence: 0.85
- Put hard limits on agent loops: maximum steps, maximum tokens and a wall-clock timeout. Confidence: 0.90
- Require human confirmation before any tool call that sends, deletes, pays or changes data outside the app. Confidence: 0.90
- Treat all retrieved documents, web pages and tool results as untrusted input that can contain prompt injection. Confidence: 0.90
- Log every model call with prompt version, model, latency, token counts and cost, with user data redacted. Confidence: 0.85
- Cache deterministic calls and use prompt caching for long, stable system prompts. Confidence: 0.75
- Use the smallest model that passes the evals, and route only hard cases to the expensive model. Confidence: 0.80
- Retry on rate limits and transient errors with exponential backoff and jitter, never on validation failures in a tight loop. Confidence: 0.85
- Keep temperature low for extraction and classification, and higher only for creative generation. Confidence: 0.75
- Show users where an answer came from and make it easy to report a bad response. Confidence: 0.80
- Collect thumbs-up and thumbs-down feedback and feed the bad cases back into the eval set. Confidence: 0.85
- Do not fine-tune until prompting, retrieval and evals have clearly hit their limits. Confidence: 0.70

# Feature Flag Rollout taste of a release engineer
- Separate deploy from release: ship code dark behind a flag and turn it on independently of the deployment. Confidence: 0.95
- Use a managed flag service or OpenFeature-compatible SDK instead of home-grown if statements on environment variables. Confidence: 0.80
- Give every flag an owner, a description, a type (release, experiment, ops or permission) and an expected removal date. Confidence: 0.90
- Name flags after the behavior they control, such as checkout-new-payment-form, not after tickets or people. Confidence: 0.85
- Default every flag to the safe, existing behavior when the flag service is unreachable. Confidence: 0.95
- Evaluate flags once per request at the edge of the code path and pass the result down, not deep inside loops. Confidence: 0.80
- Roll out gradually: internal users first, then 1%, 10%, 50% and 100%, watching metrics at each step. Confidence: 0.90
- Use sticky, hash-based bucketing on a stable user or account ID so people do not flip between variants. Confidence: 0.90
- Define the success and guardrail metrics before the rollout starts, not after looking at the dashboard. Confidence: 0.85
- Wire automatic rollback to error rate and latency alerts for risky changes. Confidence: 0.75
- Keep a kill switch for every expensive or external dependency so operations can turn it off without a deploy. Confidence: 0.90
- Remove release flags within a few weeks of reaching 100% and fail CI when a flag is past its removal date. Confidence: 0.80
- Test both the on and off paths in CI for every active flag. Confidence: 0.85
- Avoid nesting flags inside other flags; if two flags interact, merge them or document the combinations. Confidence: 0.80
- Log flag evaluations with the variant in traces so incidents can be tied to a specific rollout. Confidence: 0.80
- Restrict who can change production flags and keep an audit log of every change with a reason. Confidence: 0.85
- Never use feature flags as a substitute for authorization checks. Confidence: 0.90
- Announce large rollouts in a shared channel with the flag name, the plan and the rollback step. Confidence: 0.70
- Keep long-lived permission and ops flags in a separate list so they are not mistaken for stale release flags. Confidence: 0.75
- Run a short review after each major rollout and note which metrics actually caught problems. Confidence: 0.65

# React Native App taste of a cross-platform mobile engineer
- Start new apps with Expo and the prebuild workflow instead of a bare React Native project, and eject only when a native module truly needs it. Confidence: 0.90
- Write everything in TypeScript with strict mode on, including navigation params and native module bindings. Confidence: 0.95
- Use Expo Router for file-based navigation and keep typed routes enabled so broken links fail at compile time. Confidence: 0.80
- Keep server state in TanStack Query and limit global client state to a small Zustand store for session and preferences. Confidence: 0.85
- Never store tokens in AsyncStorage; use expo-secure-store or the Keychain and Keystore for anything sensitive. Confidence: 0.95
- Render long lists with FlashList or a well-tuned FlatList with stable keys, never a ScrollView with map. Confidence: 0.90
- Run animations on the UI thread with Reanimated and gestures with react-native-gesture-handler, not the old Animated API. Confidence: 0.85
- Enable the New Architecture and Hermes by default and treat libraries that block it as tech debt. Confidence: 0.75
- Keep a single theme module for colors, spacing and typography, and support dark mode from the first screen. Confidence: 0.85
- Respect safe areas with react-native-safe-area-context instead of hardcoded padding for notches. Confidence: 0.90
- Put platform-specific code in .ios.tsx and .android.tsx files rather than scattering Platform.OS checks. Confidence: 0.70
- Ship JavaScript fixes through EAS Update with a runtime version policy, and use store releases for native changes. Confidence: 0.80
- Build and sign in CI with EAS Build so no release depends on a single laptop. Confidence: 0.85
- Test components with React Native Testing Library and query by role and text, not by testID first. Confidence: 0.80
- Cover the critical flows end to end with Maestro or Detox on real simulators in CI. Confidence: 0.75
- Add accessibilityLabel and accessibilityRole to every touchable and keep tap targets at least 44 points. Confidence: 0.90
- Handle offline as a normal state: show cached data, queue writes and retry when the connection returns. Confidence: 0.80
- Report crashes and JavaScript errors with Sentry including source maps for every release. Confidence: 0.85
- Optimize images with expo-image and caching instead of the default Image component. Confidence: 0.80
- Profile with the React Native DevTools before guessing at performance fixes. Confidence: 0.70
- Ask for permissions only at the moment a feature needs them, with a short explanation screen first. Confidence: 0.90

# Technical Documentation taste of a developer-focused writer
- Keep docs in the same repository as the code and review them in the same pull request as the change. Confidence: 0.95
- Write in Markdown or MDX and build the site with a static generator such as Docusaurus, Starlight or MkDocs. Confidence: 0.85
- Organize content into tutorials, how-to guides, reference and explanation, following the Diataxis model. Confidence: 0.85
- Open the README with one sentence on what the project does, then install and a working quick start within the first screen. Confidence: 0.95
- Make every code sample copy-pasteable and runnable, with imports and realistic values included. Confidence: 0.95
- Test code samples in CI or extract them from tested example files so they never rot. Confidence: 0.80
- Generate API reference from the source, such as OpenAPI, TypeDoc or docstrings, instead of writing it by hand. Confidence: 0.90
- Write in second person and active voice with short sentences and plain words. Confidence: 0.90
- Use task-based headings like "Rotate an API key" rather than vague ones like "Keys". Confidence: 0.85
- State prerequisites and versions at the top of every guide. Confidence: 0.85
- Show the expected output after each command so readers know they are on track. Confidence: 0.80
- Lint prose with Vale and a shared style guide, and check links on every build. Confidence: 0.80
- Use admonitions such as Note and Warning sparingly, only for information that changes what the reader does. Confidence: 0.75
- Prefer diagrams as code with Mermaid so they can be reviewed and updated like text. Confidence: 0.75
- Keep a changelog in Keep a Changelog format and write migration guides for every breaking change. Confidence: 0.90
- Record architecture decisions as short ADRs that explain the context and the tradeoffs, not only the choice. Confidence: 0.85
- Give every page an owner and a last-reviewed date, and remove pages that are no longer true. Confidence: 0.75
- Add a search box and stable deep links to headings before adding any other site feature. Confidence: 0.80
- Write alt text for every screenshot and avoid screenshots of text that could be a code block. Confidence: 0.85
- Collect feedback with a simple "Was this page helpful?" widget and review the results monthly. Confidence: 0.65
- Document error messages by their exact text so users can find the fix by searching. Confidence: 0.85

# Machine Learning Experiment taste of an applied ML engineer
- Write the evaluation metric and a dumb baseline before training any real model. Confidence: 0.95
- Split data by time or by entity when leakage is possible, never with a random split by default. Confidence: 0.90
- Freeze the test set early and look at it only for final decisions, using a separate validation set for tuning. Confidence: 0.95
- Track every run with MLflow or Weights & Biases, logging config, metrics, code version and dataset version. Confidence: 0.90
- Keep experiment config in versioned YAML files loaded with Hydra or Pydantic, not in hardcoded constants. Confidence: 0.80
- Set and log random seeds for Python, NumPy and the framework, and report variance across several seeds. Confidence: 0.85
- Version datasets with DVC or immutable object storage paths so any result can be reproduced. Confidence: 0.80
- Prototype in notebooks but move reusable code into a tested Python package as soon as it is used twice. Confidence: 0.85
- Pin dependencies with a lock file and record the CUDA and driver versions with each run. Confidence: 0.75
- Inspect raw samples and label errors by hand before trusting any aggregate metric. Confidence: 0.90
- Prefer gradient boosted trees for tabular data and reach for deep learning only when they clearly fall short. Confidence: 0.80
- Use PyTorch with a thin training loop or Lightning instead of large custom frameworks. Confidence: 0.70
- Report metrics per slice such as region, device or class, not only a single global number. Confidence: 0.90
- Calibrate predicted probabilities when downstream code makes threshold decisions. Confidence: 0.75
- Tune hyperparameters with Optuna or a simple random search under a fixed compute budget. Confidence: 0.75
- Write unit tests for feature transforms and data loaders, and assert expected shapes and dtypes. Confidence: 0.85
- Validate input data with schema checks such as Pandera or Great Expectations before training. Confidence: 0.80
- Save the model together with its preprocessing pipeline as one artifact so training and serving cannot drift. Confidence: 0.90
- Compare a new model to the production model offline and then with a shadow or A/B rollout. Confidence: 0.85
- Monitor feature drift and prediction distributions after launch and alert on large shifts. Confidence: 0.80
- Write a short experiment note for every significant run with the hypothesis, the result and the next step. Confidence: 0.85
- Delete or archive failed branches of work instead of leaving dozens of half-finished notebooks. Confidence: 0.65

---
name: gitignore-pattern-tester-and-reviewer
description: Explains every line of a .gitignore, tests it against paths that must be ignored or must stay committable, names the rule that decides each path, and flags mistakes such as negations blocked by an excluded parent folder, inline comments, leading spaces, backslashes, stray **, secrets not covered and .env.example ignored. Returns a fixed file. Includes a stdlib Python checker with --self-test, matched against real git.
---

# Gitignore Pattern Tester and Reviewer

You help developers understand and fix `.gitignore` files. You explain what each pattern really matches, prove it with path tests, and find the mistakes that make git ignore source files or commit secrets and build output. You return a corrected file with tests that pass, not just advice.

## Files in this skill

- `scripts/gitignore_check.py` - explains each rule, runs "+ path" (must be ignored) and "- path" (must be kept) tests, names the deciding line for each path, and reports HIGH, MEDIUM, LOW and INFO findings (Python 3 standard library only)
- `references/gitignore-rules.md` - how git reads ignore files: sources and priority, line syntax, anchoring, wildcards, the parent-directory rule, tracked files and commands to confirm
- `references/common-pitfalls.md` - every finding code with its fix, pitfalls the script cannot see (broad names, tracked files, nested files, case sensitivity, lock files, .dockerignore differences) and good starting blocks
- `templates/gitignore-review.md` - the review to hand back
- `examples/webapp.gitignore` - a web app ignore file with six real bugs
- `examples/webapp-fixed.gitignore` - the corrected file (all tests pass)
- `examples/webapp-paths.txt` - 19 path tests used with both files

## Workflow

### 1. Collect the inputs
Ask for the `.gitignore` (and any nested ones), the stack, and the paths that matter: what must be ignored (build output, dependencies, logs, secrets) and what must stay committable (source folders with common names like `build/` or `lib/`, `.gitkeep` files, `.env.example`, lock files). If the user only asks "why is X ignored?" ask for the path and use `git check-ignore -v X` output if they have it.

### 2. Run the checker
```
python3 scripts/gitignore_check.py examples/webapp.gitignore --test examples/webapp-paths.txt
python3 scripts/gitignore_check.py .gitignore --test paths.txt --json
python3 scripts/gitignore_check.py --self-test
```
Paths file: `+ path` must be ignored, `- path` must be kept, end directories with `/`. Exit 1 means a HIGH finding or a failed test, 2 means unreadable input. The checker models one ignore file at the repo root; for a nested file, give paths relative to its folder.

### 3. Explain and fix
- Walk through the Rules table in plain words, especially anchoring (slash at start or middle = relative to the .gitignore folder) and the parent-directory rule (see `references/gitignore-rules.md`).
- For each FAIL, quote the deciding line from the "Decided by" column and give the smallest change that fixes it.
- Fix every HIGH and MEDIUM finding; explain any INFO you leave in place.
- Add `- path` tests for source folders with common names so over-broad rules show up.
- Re-run the checker on the fixed file until all tests pass, and show the diff.

### 4. Deliver
Fill `templates/gitignore-review.md`: what the current file does, the test table, findings, the full fixed file, a diff, and the commands to confirm (`git check-ignore -v`, `git status --ignored`) and to untrack files that were already committed (`git rm --cached`).

## Example result
For `examples/webapp.gitignore` with `examples/webapp-paths.txt`: 13/19 tests pass, 1 HIGH, 4 MEDIUM, 2 LOW, 1 INFO. HIGH NEGATION_BLOCKED: `logs/` excludes the folder, so `!logs/.gitkeep` can never apply (fix: `logs/*`). MEDIUM: `config/secrets.yml # never commit` is one pattern, so the secrets file would be committed; ` Thumbs.db` has a leading space; `src\generated` is read as `srcgenerated`; and `*.key`, `*.pem` and `id_rsa` files are not covered. LOW: `.env*` hides `.env.example`; `*.log` is listed twice. The unanchored `build` also hides `src/build/README.md` (fix: `/build/`). `examples/webapp-fixed.gitignore` passes 19/19 with no HIGH, MEDIUM or LOW findings.

## Rules

- Last matching line wins; a file inside an excluded folder can never be re-included. Say this whenever a `!` line is involved.
- Never suggest ignoring lock files (`package-lock.json`, `poetry.lock`, `go.sum`, app `Cargo.lock`) or shared config the team commits on purpose.
- If a secret was already committed, say that ignoring it is not enough: rotate it and consider rewriting history.
- Prefer anchored root patterns (`/dist/`, `/build/`) for output folders, and put personal editor and OS files in the global ignore file when the team agrees.
- Confirm on the real repository with `git check-ignore -v`: the script follows git's documented rules for one file, but nested files, `.git/info/exclude`, the global file and `core.ignorecase` can change the result.
FILE:references/gitignore-rules.md
# How git reads .gitignore

Source: the gitignore(5) manual (`git help gitignore`). The checker implements these rules and was compared with real git 2.47 on 6,956 path checks with no differences.

## Where rules come from (highest priority first)
1. Command-line `--exclude` options (rare; used by some tools).
2. `.gitignore` files: the one in the same folder as the path wins over the ones above it. Patterns are relative to the folder that holds that `.gitignore`.
3. `.git/info/exclude` - local to one clone, never committed. Good for personal scratch files.
4. The global file in `core.excludesFile` (default `~/.config/git/ignore`) - good for editor and OS files such as `.DS_Store` and `*.swp`.

Inside one file, the LAST matching line decides.

## Line syntax
| Line | Meaning |
|---|---|
| empty | ignored, use as a separator |
| `# text` | comment; only at the very start of a line. `foo # bar` is the pattern `foo # bar` |
| `\#file` | a name that starts with `#` |
| `!pattern` | re-include something an earlier line excluded |
| `\!file` | a name that starts with `!` |
| `name   ` | trailing spaces are dropped unless escaped (`name\ `) |
| `  name` | leading spaces are KEPT and become part of the name |
| `\` | escapes the next character; it is never a path separator |

## Slashes decide where a pattern matches
| Pattern | Matches | Does not match |
|---|---|---|
| `build` | `build`, `src/build`, `a/b/build/x.js` (any file or folder named build) | `build.sh` |
| `build/` | folders named build at any depth | a FILE named build |
| `/build` | `build` next to the .gitignore only | `src/build` |
| `doc/frotz` | `doc/frotz` next to the .gitignore (a slash in the middle anchors like a leading slash) | `a/doc/frotz` |
| `**/doc/frotz` | `doc/frotz` at any depth | |

Rule of thumb: no slash (or only a trailing one) = any depth; a slash at the start or middle = relative to the .gitignore folder.

## Wildcards
| Wildcard | Meaning |
|---|---|
| `*` | any characters except `/` (`*.log`, `/a*z` does not match `ab/cz`) |
| `?` | exactly one character except `/` |
| `[abc]`, `[a-z]`, `[!0-9]`, `[^0-9]` | one character from (or not from) the set; `[[:digit:]]` style classes work |
| `**/name` | `name` in any folder, including the top |
| `dir/**` | everything inside `dir`, but not `dir` itself |
| `a/**/b` | `a/b`, `a/x/b`, `a/x/y/b` |
| `foo**`, `**bar` | `**` not next to slashes acts like a single `*` |

## The parent-directory rule (the most common surprise)
"It is not possible to re-include a file if a parent directory of that file is excluded." Git does not even list the contents of an excluded folder, so a later `!` line never gets a chance.

```gitignore
# does NOT work: logs/ is excluded as a folder
logs/
!logs/.gitkeep

# works: exclude the folder's contents, keep the folder visible
logs/*
!logs/.gitkeep

# works for deeper files: re-include each folder level
/assets/*
!/assets/icons/
/assets/icons/*
!/assets/icons/app.svg
```

## Whitelist style
Ignore everything, then allow what you want:
```gitignore
*
!*/
!*.md
!src/**
```
`!*/` re-includes every folder so the later `!` lines can reach files inside them.

## Already tracked files
.gitignore only affects untracked files. A file that is already committed stays tracked:
```
git rm --cached path/to/file        # keep it on disk, stop tracking it
git rm -r --cached build/           # same for a folder
```
If it was a secret, removing it from tracking is not enough: rotate the secret, because it is still in history.

## Commands to confirm
```
git check-ignore -v path/to/file          # prints the file, line and pattern that ignores it (nothing = not ignored)
git check-ignore -v --no-index path       # also for files that are tracked
git status --ignored --short              # lists ignored files as "!!"
git ls-files --others --ignored --exclude-standard   # every ignored untracked file
```
FILE:references/common-pitfalls.md
# Common .gitignore pitfalls and the checker's codes

| Code | Severity | What goes wrong | Fix |
|---|---|---|---|
| NEGATION_BLOCKED | HIGH | `!path/file` after a rule that excludes a parent folder (`logs/`, `dist`, `a/*` for `a/b/c`) never takes effect, so the file you meant to commit stays ignored | Exclude contents (`logs/*` or `logs/**`) instead of the folder, or re-include each folder level (`!a/b/`) |
| SECRETS_NOT_COVERED | MEDIUM | typical secret files (`.env`, `*.pem`, `*.key`, `id_rsa`) would be committed by `git add .` | Add `.env`, `.env.*`, `*.pem`, `*.key`, `id_rsa*`; also use a secret scanner in CI |
| INLINE_COMMENT | MEDIUM | `secrets.yml # never commit` is one pattern including the comment text, so `secrets.yml` is NOT ignored | Put comments on their own line |
| LEADING_SPACE | MEDIUM | ` Thumbs.db` matches only a name that starts with a space | Remove the space |
| BACKSLASH_SEPARATOR | MEDIUM | `src\generated` means "src" + escaped "g" + "enerated" = `srcgenerated` | Always use `/`, also on Windows |
| ENV_EXAMPLE_IGNORED | LOW | `.env*` also hides `.env.example`, the template other developers need | Add `!.env.example` after it |
| DUPLICATE | LOW | the same line twice; harmless but a sign of copy-paste merges | Delete one |
| DOUBLE_STAR | LOW | `foo**` is just `foo*`; the author probably meant `foo/**` | Use `**` only as `**/x`, `x/**` or `a/**/b` |
| TRAILING_SPACE | LOW | trailing spaces are silently dropped; if they were meant, the file will not match | Remove or escape them |
| CRLF | LOW | Windows line endings in .gitignore | Save as LF; add `* text=auto` to .gitattributes |
| ANCHORED_BY_SLASH | INFO | `src/gen` only matches next to the .gitignore, not `packages/a/src/gen` | Add `/` to show intent or `**/` to match at any depth |

## Pitfalls the script cannot see (check them by hand)
- **Unanchored names that are too broad.** `build` also hides `src/build/` and `docs/build/`; `lib` hides every `lib` folder including source code. Use `/build/` for the root output folder. Write a `- path` test for any source folder with a common name.
- **Tracked files.** Adding a rule does not untrack files already committed. Run `git ls-files -ci --exclude-standard` to list tracked files that match the ignore rules, then `git rm --cached`.
- **Nested .gitignore files.** A `.gitignore` in a subfolder adds rules relative to that subfolder and can re-include what the root file excluded (unless a parent folder is excluded). Run the checker on each file with paths relative to its folder, or confirm with `git check-ignore -v`.
- **Case sensitivity.** On macOS and Windows `core.ignorecase=true` makes `*.JPG` match `photo.jpg`; on Linux CI it does not. Write both cases or use `*.[jJ][pP][gG]`.
- **Personal files in the shared file.** `.idea/`, `.vscode/`, `*.swp`, `.DS_Store` belong in the global ignore file (`core.excludesFile`), though many teams keep them in the repo file for convenience. Do not ignore `.vscode/settings.json` if the team shares it on purpose.
- **Lock files.** Do not ignore `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `poetry.lock`, `Cargo.lock` (for applications) or `go.sum`: they make builds reproducible.
- **Ignoring then committing anyway.** `git add -f` overrides ignore rules; reviewers should check for force-added build output or secrets.
- **Other ignore files use different rules.** `.dockerignore` uses Go filepath.Match plus `**` and is always anchored at the build context root (no "any depth" for plain names). `.npmignore`, `.eslintignore` and `.prettierignore` follow gitignore-like rules but not always every detail. Test them with their own tools.

## Good starting blocks
```gitignore
# Node
node_modules/
/dist/
/build/
npm-debug.log*
.npm/

# Python
__pycache__/
*.py[cod]
.venv/
.pytest_cache/
/dist/
*.egg-info/

# Secrets
.env
.env.*
!.env.example
*.pem
*.key
id_rsa*

# Logs that keep their folder
logs/*
!logs/.gitkeep
```
FILE:templates/gitignore-review.md
# .gitignore review: <repository or folder>

**Stack:** <Node 20 + Vite / Python 3.12 / monorepo with pnpm workspaces>
**Files reviewed:** <.gitignore at root, packages/api/.gitignore>
**Summary:** <one sentence: the most important problem and what the fix changes>

## What the current file does
| Line | Pattern | Meaning |
|---|---|---|
| <9> | <`logs/`> | <ignores every folder named logs, at any depth, and everything in it> |

## Path tests
| Result | Path | Expected | Actual | Decided by |
|---|---|---|---|---|
| <FAIL> | <`logs/.gitkeep`> | <kept> | <ignored> | <parent logs/ excluded by L9; L10 cannot re-include it> |

## Findings
| # | Severity | Code | Line | Problem | Fix |
|---|---|---|---|---|---|
| 1 | <HIGH> | <NEGATION_BLOCKED> | <10> | <...> | <use logs/* instead of logs/> |

## Proposed .gitignore
```gitignore
<full fixed file, with comments on their own lines>
```

## Changes
```diff
- logs/
+ logs/*
```

## Apply and confirm
1. Replace the file, then run: `git check-ignore -v <each path from the tests>`
2. `git status --ignored --short` and check nothing important shows as `!!`
3. Untrack files that are now ignored but were committed: `git rm --cached <path>` (rotate any secret that was committed)
4. Commit: `chore: fix .gitignore (<short reason>)`

## Follow-ups
- <move editor files to the global ignore file>
- <add a secret scanner (gitleaks, trufflehog) to CI>
FILE:examples/webapp.gitignore
# Dependencies
node_modules
/vendor/

# Build output
dist/
build
*.log
logs/
!logs/.gitkeep

# Environment
.env*
config/secrets.yml # never commit

# OS and editor files
.DS_Store
.idea/
*.swp
 Thumbs.db

# Test coverage
coverage/**
!coverage/README.md

# Generated code
src\generated
*.log
FILE:examples/webapp-fixed.gitignore
# Dependencies
node_modules/
/vendor/

# Build output (root only, so src/build/ stays tracked)
/dist/
/build/
*.log
# ignore the contents of logs/, not the folder, so the negation can work
logs/*
!logs/.gitkeep

# Environment and secrets
.env
.env.*
!.env.example
*.pem
*.key
id_rsa*
# never commit
config/secrets.yml

# OS and editor files
.DS_Store
Thumbs.db
.idea/
*.swp

# Test coverage
coverage/**
!coverage/README.md

# Generated code
/src/generated/
FILE:examples/webapp-paths.txt
# Paths for examples/webapp.gitignore and webapp-fixed.gitignore
# "+ path" = must be ignored, "- path" = must stay committable
+ node_modules/react/index.js
+ packages/api/node_modules/lodash/lodash.js
+ vendor/autoload.php
- packages/vendor/README.md
+ dist/app.js
+ build/output.js
- src/build/README.md
+ logs/app.log
- logs/.gitkeep
+ .env
+ .env.local
- .env.example
+ config/secrets.yml
+ .DS_Store
+ Thumbs.db
+ coverage/lcov.info
- coverage/README.md
+ src/generated/types.ts
- src/index.ts
FILE:scripts/gitignore_check.py
#!/usr/bin/env python3
"""Gitignore pattern tester and reviewer (Python 3 standard library only).

Explains every rule in a .gitignore file, tests the file against paths that must be
ignored ("+ path") or must stay committable ("- path"), names the rule that decides
each path, and flags common mistakes: negations that cannot work because a parent
directory is excluded, inline comments, leading spaces, Windows backslashes, stray
"**", duplicates, secrets that are not covered and .env.example being ignored.

Matching follows git's documented gitignore rules for a single .gitignore at the
repository root: last matching rule wins, "!" re-includes, a trailing "/" matches
directories only, a "/" at the start or in the middle anchors the pattern to the
.gitignore's directory, "*" "?" "[...]" never match "/", "**" spans directories,
and a file can never be re-included if a parent directory is excluded.

Usage:
  python3 gitignore_check.py .gitignore [--test paths.txt] [--json]
  python3 gitignore_check.py --self-test
Paths file: one path per line, "+ path" = must be ignored, "- path" = must NOT be
ignored, end a path with "/" if it is a directory, "#" starts a comment.
Exit codes: 0 = no HIGH findings and all tests pass, 1 = HIGH finding or failed test,
2 = unreadable input.
"""
import argparse, json, re, sys

SEV_ORDER = {"HIGH": 0, "MEDIUM": 1, "LOW": 2, "INFO": 3}
SECRET_PROBES = [".env", "config/.env.production", "certs/server.key", "deploy/id_rsa",
                 "secrets/service-account.pem"]
POSIX_CLASSES = {"alnum": "a-zA-Z0-9", "alpha": "a-zA-Z", "digit": "0-9", "lower": "a-z",
                 "upper": "A-Z", "space": " \\t\\n\\r\\f\\v", "xdigit": "0-9A-Fa-f",
                 "punct": re.escape("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"), "blank": " \\t"}


class Rule:
    def __init__(self, line_no, raw, negate, dir_only, anchored, body, regex):
        self.line_no, self.raw, self.negate, self.dir_only = line_no, raw, negate, dir_only
        self.anchored, self.body, self.regex = anchored, body, regex

    def matches(self, rel_path, is_dir):
        if self.dir_only and not is_dir:
            return False
        target = rel_path if self.anchored else rel_path.rsplit("/", 1)[-1]
        return self.regex.fullmatch(target) is not None

    def label(self):
        return f"L{self.line_no} `{self.raw}`"


def strip_trailing_spaces(s):
    """git drops trailing spaces unless the last one is escaped with a backslash."""
    end = len(s)
    while end > 0 and s[end - 1] == " ":
        bs, k = 0, end - 2
        while k >= 0 and s[k] == "\\":
            bs += 1; k -= 1
        if bs % 2 == 1:
            break
        end -= 1
    return s[:end]


def class_to_regex(body):
    out, m = "", 0
    while m < len(body):
        ch = body[m]
        if ch == "[" and body.startswith("[:", m):
            end = body.find(":]", m + 2)
            if end != -1 and body[m + 2:end] in POSIX_CLASSES:
                out += POSIX_CLASSES[body[m + 2:end]]; m = end + 2; continue
        if ch == "\\" and m + 1 < len(body):
            out += re.escape(body[m + 1]); m += 2; continue
        if ch == "-" and 0 < m < len(body) - 1:
            out += "-"; m += 1; continue
        out += re.escape(ch); m += 1
    return out


def glob_to_regex(p):
    out, i, n = "", 0, len(p)
    while i < n:
        c = p[i]
        if c == "\\":
            if i + 1 < n:
                out += re.escape(p[i + 1]); i += 2
            else:
                i += 1  # a lone trailing backslash matches nothing extra
            continue
        if c == "*":
            j = i
            while j < n and p[j] == "*":
                j += 1
            if j - i >= 2 and (i == 0 or p[i - 1] == "/") and (j == n or p[j] == "/"):
                if j == n:
                    out += ".*"; i = j
                else:
                    out += "(?:.*/)?"; i = j + 1
                continue
            out += "[^/]*"; i = j; continue
        if c == "?":
            out += "[^/]"; i += 1; continue
        if c == "[":
            j = i + 1
            neg = j < n and p[j] in "!^"
            if neg:
                j += 1
            k = j + 1 if j < n and p[j] == "]" else j
            while k < n and p[k] != "]":
                if p[k] == "[" and p.startswith("[:", k):
                    end = p.find(":]", k + 2)
                    if end != -1:
                        k = end + 2; continue
                k += 2 if p[k] == "\\" else 1
            if k >= n:
                out += re.escape("["); i += 1; continue
            cls = class_to_regex(p[j:k])
            out += ("[^/" + cls + "]") if neg else ("(?!/)[" + cls + "]")
            i = k + 1; continue
        out += re.escape(c); i += 1
    return out


def parse_rules(text):
    rules, findings, seen = [], [], {}
    for no, raw in enumerate(text.split("\n"), 1):
        if raw.endswith("\r"):
            findings.append(("LOW", "CRLF", no, "line ends with CR (Windows line ending)",
                             "Save the file with LF line endings; set `* text=auto` in .gitattributes."))
            raw = raw[:-1]
        if not raw.strip() or raw.startswith("#"):
            continue
        line = strip_trailing_spaces(raw)
        if line != raw:
            findings.append(("LOW", "TRAILING_SPACE", no, f"`{raw}` has trailing spaces, which git drops",
                             "Delete them; escape a real trailing space as `\\ `."))
        if raw[0] in " \t":
            findings.append(("MEDIUM", "LEADING_SPACE", no, f"`{raw}` starts with whitespace, which git keeps as part of the name, so it matches nothing useful",
                             "Remove the leading whitespace."))
        m = re.search(r"\s#", line)
        if m:
            findings.append(("MEDIUM", "INLINE_COMMENT", no, f"`{line}`: git has no inline comments, so `{line[m.start():].strip()}` becomes part of the pattern",
                             "Move the comment to its own line starting with #."))
        if re.search(r"\\[A-Za-z0-9_]", line):
            findings.append(("MEDIUM", "BACKSLASH_SEPARATOR", no, f"`{line}` uses a backslash, which escapes the next character instead of separating directories",
                             "Use forward slashes: `" + line.replace("\\", "/") + "`."))
        key = line.strip()
        if key in seen:
            findings.append(("LOW", "DUPLICATE", no, f"`{key}` repeats line {seen[key]}", "Delete the duplicate."))
        else:
            seen[key] = no
        negate = line.startswith("!")
        body = line[1:] if negate else line
        if body.startswith("\\!") or body.startswith("\\#"):
            body = body[1:]
        if negate and not body:
            findings.append(("LOW", "EMPTY_NEGATION", no, "`!` alone does nothing", "Delete the line."))
            continue
        dir_only = body.endswith("/") and not body.endswith("\\/")
        if dir_only:
            body = body.rstrip("/")
        if not body:
            continue
        anchored = "/" in body
        if body.startswith("/"):
            body = body[1:]
        if any((m.start() > 0 and body[m.start() - 1] != "/") or (m.end() < len(body) and body[m.end()] != "/")
               for m in re.finditer(r"\*{2,}", body)):
            findings.append(("LOW", "DOUBLE_STAR", no, f"`{line}`: `**` not between slashes acts like a single `*`",
                             "Use `**/name`, `dir/**` or `a/**/b`, or a single `*`."))
        rules.append(Rule(no, line, negate, dir_only, anchored, body, re.compile(glob_to_regex(body), re.S)))
    return rules, findings


def last_match(rules, rel_path, is_dir):
    hit = None
    for r in rules:
        if r.matches(rel_path, is_dir):
            hit = r
    return hit


def check_path(rules, path):
    """Return (ignored, reason, blocked_rule) following git's parent-directory rule."""
    is_dir = path.endswith("/")
    parts = [p for p in path.strip("/").split("/") if p]
    for k in range(1, len(parts) + 1):
        sub = "/".join(parts[:k])
        sub_is_dir = k < len(parts) or is_dir
        hit = last_match(rules, sub, sub_is_dir)
        if k < len(parts):
            if hit and not hit.negate:
                own = last_match(rules, "/".join(parts), is_dir)
                blocked = own if own and own.negate else None
                return True, f"parent `{sub}/` excluded by {hit.label()}", blocked
        else:
            if hit is None:
                return False, "no rule matches", None
            if hit.negate:
                return False, f"re-included by {hit.label()}", None
            return True, f"ignored by {hit.label()}", None
    return False, "empty path", None


def literal_dirs(body):
    parts = body.split("/")[:-1]
    out = []
    for p in parts:
        if any(ch in p for ch in "*?[\\"):
            break
        out.append(p)
    return out


def lint_negations(rules):
    findings = []
    for idx, r in enumerate(rules):
        if not r.negate or not r.anchored:
            continue
        before = rules[:idx]
        dirs = literal_dirs(r.body)
        for k in range(1, len(dirs) + 1):
            sub = "/".join(dirs[:k])
            hit = last_match(before, sub, True)
            if hit and not hit.negate:
                findings.append(("HIGH", "NEGATION_BLOCKED", r.line_no,
                                 f"`{r.raw.strip()}` can never apply: its parent directory `{sub}/` is excluded by {hit.label()} and git does not look inside excluded directories",
                                 f"Exclude the contents instead of the directory (`{sub}/*` or `{sub}/**`), then keep the `!` rule; or re-include `!{sub}/` first."))
                break
    return findings


def lint_secrets(rules):
    findings = []
    missing = [p for p in SECRET_PROBES if not check_path(rules, p)[0]]
    if missing:
        findings.append(("MEDIUM", "SECRETS_NOT_COVERED", 0, "these typical secret files would NOT be ignored: " + ", ".join(missing),
                         "Add `.env`, `.env.*` (with `!.env.example`), `*.pem`, `*.key`, `id_rsa*` if the project could ever contain them."))
    if check_path(rules, ".env.example")[0]:
        findings.append(("LOW", "ENV_EXAMPLE_IGNORED", 0, "`.env.example` is ignored, so the documented template cannot be committed",
                         "Add `!.env.example` after the .env rule."))
    return findings


def explain(rule):
    body = rule.body
    kind = "directories" if rule.dir_only else "files or directories"
    verb = "Re-includes" if rule.negate else "Ignores"
    wild = any(ch in body for ch in "*?[")
    if rule.anchored and body.endswith("/**") and "*" not in body[:-3]:
        text = f"{verb} everything inside `{body[:-3]}/` (but not the folder itself), relative to the .gitignore folder"
    elif rule.anchored and body.startswith("**/") and "/" not in body[3:]:
        text = f"{verb} {kind} {'matching' if wild else 'named'} `{body[3:]}` at any depth"
    elif rule.anchored:
        text = f"{verb} {kind} {'matching' if wild else 'at'} `{body}` relative to the .gitignore folder only"
    else:
        text = f"{verb} {kind} {'matching' if wild else 'named'} `{body}` at any depth"
    if not rule.negate and not body.endswith("/**"):
        text += "; a matched directory hides everything inside it"
    if rule.negate:
        text += " (only works if no parent directory is excluded)"
    return text + "."


def read_tests(text):
    tests = []
    for no, line in enumerate(text.splitlines(), 1):
        s = line.strip()
        if not s or s.startswith("#"):
            continue
        if s[0] not in "+-" or len(s) < 3:
            raise ValueError(f"paths file line {no}: expected '+ path' or '- path', got {line!r}")
        tests.append((s[0] == "+", s[1:].strip()))
    return tests


def review(gi_text, tests_text=None):
    rules, findings = parse_rules(gi_text)
    findings += lint_negations(rules)
    findings += lint_secrets(rules)
    anchored = [r.line_no for r in rules if r.anchored and not r.negate and "/" in r.body
                and not r.raw.startswith(("/", "**/")) and not r.body.endswith("/**")]
    if anchored:
        findings.append(("INFO", "ANCHORED_BY_SLASH", anchored[0],
                         "lines " + ", ".join(map(str, anchored)) + " contain a slash, so they only match relative to the .gitignore folder, not deeper copies",
                         "Fine if intended; add a leading `/` to make that explicit, or prefix `**/` to match at any depth."))
    results = []
    if tests_text:
        for want, path in read_tests(tests_text):
            got, reason, blocked = check_path(rules, path)
            note = reason
            if blocked:
                note += f"; {blocked.label()} cannot re-include it"
            results.append({"path": path, "expect": "ignored" if want else "kept",
                            "actual": "ignored" if got else "kept", "pass": got == want, "why": note})
    findings.sort(key=lambda f: (SEV_ORDER[f[0]], f[2]))
    return {"rules": [{"line": r.line_no, "pattern": r.raw, "meaning": explain(r)} for r in rules],
            "findings": [dict(zip(("severity", "code", "line", "problem", "fix"), f)) for f in findings],
            "tests": results}


def render(name, rep):
    out = [f"# .gitignore review: {name}", ""]
    counts = {s: sum(1 for f in rep["findings"] if f["severity"] == s) for s in SEV_ORDER}
    failed = sum(1 for t in rep["tests"] if not t["pass"])
    out.append(f"Findings: {counts['HIGH']} HIGH, {counts['MEDIUM']} MEDIUM, {counts['LOW']} LOW, {counts['INFO']} INFO")
    if rep["tests"]:
        out.append(f"Tests: {len(rep['tests']) - failed}/{len(rep['tests'])} passed")
    out += ["", "## Rules", "", "| Line | Pattern | Meaning |", "|---|---|---|"]
    for r in rep["rules"]:
        out.append(f"| {r['line']} | `{r['pattern']}` | {r['meaning']} |")
    if rep["tests"]:
        out += ["", "## Path tests", "", "| Result | Path | Expected | Actual | Decided by |", "|---|---|---|---|---|"]
        for t in rep["tests"]:
            out.append(f"| {'PASS' if t['pass'] else 'FAIL'} | `{t['path']}` | {t['expect']} | {t['actual']} | {t['why']} |")
    out += ["", "## Findings", ""]
    if rep["findings"]:
        out += ["| Severity | Code | Line | Problem | Fix |", "|---|---|---|---|---|"]
        for f in rep["findings"]:
            out.append(f"| {f['severity']} | {f['code']} | {f['line'] or '-'} | {f['problem']} | {f['fix']} |")
    else:
        out.append("No findings.")
    out += ["", "Confirm on the real repo with `git check-ignore -v <path>` and `git status --ignored`; files that are already tracked stay tracked until `git rm --cached <path>`."]
    return "\n".join(out)


def self_test():
    cases = []
    def t(name, cond):
        cases.append((name, bool(cond)))
    R = lambda s: parse_rules(s)[0]
    ign = lambda s, p: check_path(R(s), p)[0]
    t("*.log matches at any depth", ign("*.log", "a/b/c.log") and ign("*.log", "x.log"))
    t("* does not cross /", not ign("/a*z", "ab/cz") and ign("/a*z", "abcz"))
    t("name without slash matches basename anywhere", ign("build", "src/build/x.js"))
    t("leading slash anchors", ign("/vendor", "vendor/a") and not ign("/vendor", "lib/vendor/a"))
    t("middle slash anchors", ign("doc/frotz", "doc/frotz") and not ign("doc/frotz", "a/doc/frotz"))
    t("trailing slash = directory only", ign("logs/", "logs/x") and not ign("logs/", "logs"))
    t("dir-only rule ignores a dir path", ign("logs/", "logs/"))
    t("**/ prefix matches any depth", ign("**/foo", "foo") and ign("**/foo", "a/b/foo"))
    t("**/foo/bar", ign("**/foo/bar", "x/foo/bar") and not ign("**/foo/bar", "x/foo/baz"))
    t("trailing /** matches contents only", ign("abc/**", "abc/x/y") and not ign("abc/**", "abc"))
    t("a/**/b zero or more dirs", ign("a/**/b", "a/b") and ign("a/**/b", "a/x/y/b") and not ign("a/**/b", "ab"))
    t("? matches one char not /", ign("/a?c", "abc") and not ign("/a?c", "a/c"))
    t("[abc] class", ign("file[12].txt", "file1.txt") and not ign("file[12].txt", "file3.txt"))
    t("[!x] negated class", ign("f[!0-9]", "fa") and not ign("f[!0-9]", "f5"))
    t("[[:digit:]] posix class", ign("v[[:digit:]]", "v7") and not ign("v[[:digit:]]", "vx"))
    t("last match wins with negation", not ign("*.txt\n!keep.txt", "keep.txt") and ign("*.txt\n!keep.txt", "a.txt"))
    t("order matters", ign("!keep.txt\n*.txt", "keep.txt"))
    t("parent dir excluded blocks negation", ign("logs/\n!logs/.gitkeep", "logs/.gitkeep"))
    t("dir/* allows negation", not ign("logs/*\n!logs/.gitkeep", "logs/.gitkeep"))
    t("dir/** allows negation", not ign("coverage/**\n!coverage/README.md", "coverage/README.md"))
    t("re-include dir then file", not ign("/a/*\n!/a/b/\n/a/b/*\n!/a/b/c", "a/b/c"))
    t("escaped # and !", ign("\\#notes", "#notes") and ign("\\!bang", "!bang"))
    t("comment lines skipped", not ign("# *.log", "a.log"))
    t("trailing spaces stripped", ign("foo   ", "foo"))
    t("escaped trailing space kept", ign("foo\\ ", "foo ") and not ign("foo\\ ", "foo"))
    t("leading space kept as part of name", not ign(" Thumbs.db", "Thumbs.db"))
    t("inline comment becomes pattern", not ign("secrets.yml # x", "secrets.yml"))
    t("backslash escapes, not a separator", not ign("src\\gen", "src/gen/a.ts"))
    def codes(s):
        return {f[1] for f in parse_rules(s)[1]} | {f[1] for f in lint_negations(parse_rules(s)[0])} | {f[1] for f in lint_secrets(parse_rules(s)[0])}
    t("NEGATION_BLOCKED found", "NEGATION_BLOCKED" in codes("logs/\n!logs/.gitkeep"))
    t("NEGATION_BLOCKED not for dir/*", "NEGATION_BLOCKED" not in codes("logs/*\n!logs/.gitkeep"))
    t("NEGATION_BLOCKED via dir/* deeper", "NEGATION_BLOCKED" in codes("a/*\n!a/b/c.txt"))
    t("INLINE_COMMENT found", "INLINE_COMMENT" in codes("x.yml # c"))
    t("LEADING_SPACE found", "LEADING_SPACE" in codes(" x"))
    t("BACKSLASH_SEPARATOR found", "BACKSLASH_SEPARATOR" in codes("a\\b"))
    t("DUPLICATE found", "DUPLICATE" in codes("*.log\n*.log"))
    t("DOUBLE_STAR found", "DOUBLE_STAR" in codes("foo**") and "DOUBLE_STAR" not in codes("**/foo\nfoo/**"))
    t("CRLF found", "CRLF" in codes("a\r\nb"))
    t("SECRETS_NOT_COVERED found", "SECRETS_NOT_COVERED" in codes("*.log"))
    t("secrets covered", "SECRETS_NOT_COVERED" not in codes(".env\n.env.*\n!.env.example\n*.key\n*.pem\nid_rsa*"))
    t("ENV_EXAMPLE_IGNORED found", "ENV_EXAMPLE_IGNORED" in codes(".env*"))
    acodes = lambda s: {f["code"] for f in review(s)["findings"]}
    t("ANCHORED_BY_SLASH info", "ANCHORED_BY_SLASH" in acodes("src/gen") and "ANCHORED_BY_SLASH" not in acodes("/src/gen\nlogs/**\n**/x/y"))
    rep = review("logs/\n!logs/.gitkeep\n", "+ logs/a.log\n- logs/.gitkeep\n")
    t("test table marks blocked negation FAIL", rep["tests"][0]["pass"] and not rep["tests"][1]["pass"] and "cannot re-include" in rep["tests"][1]["why"])
    try:
        read_tests("? bad")
        t("bad paths line rejected", False)
    except ValueError:
        t("bad paths line rejected", True)
    t("findings sorted HIGH first", review("x # c\nlogs/\n!logs/a\n")["findings"][0]["severity"] == "HIGH")
    passed = sum(ok for _, ok in cases)
    for name, ok in cases:
        print(("PASS " if ok else "FAIL ") + name)
    print(f"{passed}/{len(cases)} passed")
    return 0 if passed == len(cases) else 1


def main():
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("gitignore", nargs="?")
    ap.add_argument("--test", help="paths file with '+ path' (must be ignored) and '- path' (must be kept)")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args()
    if a.self_test:
        return self_test()
    if not a.gitignore:
        ap.print_usage(sys.stderr); return 2
    try:
        gi = open(a.gitignore, encoding="utf-8", newline="").read()
        tests = open(a.test, encoding="utf-8").read() if a.test else None
        rep = review(gi, tests)
    except (OSError, UnicodeDecodeError, ValueError) as e:
        print(f"error: {e}", file=sys.stderr); return 2
    print(json.dumps(rep, indent=2) if a.json else render(a.gitignore, rep))
    bad = any(f["severity"] == "HIGH" for f in rep["findings"]) or any(not t["pass"] for t in rep["tests"])
    return 1 if bad else 0


if __name__ == "__main__":
    sys.exit(main())

# Background Jobs taste of a backend reliability engineer
- Make every job idempotent so a retry or duplicate delivery never charges a card or sends an email twice. Confidence: 0.95
- Pass IDs in the job payload and load fresh data inside the worker instead of serializing whole records. Confidence: 0.90
- Enqueue jobs in the same database transaction as the write that triggers them, using an outbox table or a Postgres-backed queue. Confidence: 0.85
- Prefer a boring, well-supported queue such as Sidekiq, BullMQ, Celery or a Postgres queue like Graphile Worker over a home-grown one. Confidence: 0.80
- Retry with exponential backoff and jitter, and cap attempts so poison jobs end up in a dead letter queue. Confidence: 0.95
- Classify errors as retryable or permanent and fail permanent ones immediately instead of burning retries. Confidence: 0.85
- Set an explicit timeout on every job and on every network call it makes. Confidence: 0.90
- Keep jobs small and short, and split long batch work into chunks that can resume from a checkpoint. Confidence: 0.85
- Use separate queues for latency-sensitive and bulk work so a backfill never delays password reset emails. Confidence: 0.90
- Rate limit jobs that call third-party APIs per provider, not per worker process. Confidence: 0.80
- Use unique job keys or locks to prevent the same scheduled job from running twice at once. Confidence: 0.85
- Run cron-style schedules from one scheduler with leader election, never from every app instance. Confidence: 0.80
- Version job payloads and keep workers able to read the previous version during deploys. Confidence: 0.80
- Handle SIGTERM gracefully: stop taking new jobs, finish or requeue the current one, then exit. Confidence: 0.90
- Log a job ID, attempt number and correlation ID on every log line from a job. Confidence: 0.85
- Alert on queue depth, oldest job age and dead letter growth, not just on worker crashes. Confidence: 0.90
- Give the dead letter queue an owner and a documented replay command. Confidence: 0.75
- Test jobs by calling the handler directly in unit tests and run one end-to-end test through the real queue in CI. Confidence: 0.80
- Avoid chaining many jobs into hidden workflows; reach for a workflow engine like Temporal when steps need orchestration. Confidence: 0.70
- Store job results and side-effect markers in the database, not only in the queue backend. Confidence: 0.75

# Vue App taste of a frontend engineer
- Start new projects with Vite and Vue 3, or Nuxt when server rendering or file-based routing is needed. Confidence: 0.90
- Write components with script setup and the Composition API, not the Options API. Confidence: 0.90
- Use TypeScript everywhere and check templates with vue-tsc in CI. Confidence: 0.90
- Type props and emits with defineProps and defineEmits generics instead of runtime-only declarations. Confidence: 0.85
- Use Pinia for shared client state and keep stores small and focused on one domain. Confidence: 0.85
- Fetch server data with TanStack Query for Vue or Nuxt useFetch instead of hand-rolled loading flags in stores. Confidence: 0.80
- Extract reusable logic into composables named useSomething that return refs and functions. Confidence: 0.90
- Reach for VueUse before writing a custom composable for common browser APIs. Confidence: 0.75
- Prefer computed over watch, and use watch only for side effects. Confidence: 0.90
- Never mutate props; emit events or use defineModel for two-way binding. Confidence: 0.95
- Always give v-for items a stable key from the data, never the index. Confidence: 0.90
- Never combine v-if and v-for on the same element. Confidence: 0.90
- Keep components under about 200 lines and split them when the template gets hard to scan. Confidence: 0.70
- Use multi-word PascalCase component names and colocate components by feature, not by type. Confidence: 0.80
- Style with scoped CSS or Tailwind, and avoid global styles beyond a small reset and design tokens. Confidence: 0.75
- Lazy load route components with dynamic imports in Vue Router. Confidence: 0.85
- Test components with Vitest and Vue Testing Library, querying by role and text. Confidence: 0.85
- Lint with ESLint and eslint-plugin-vue using the recommended rules, and format with Prettier. Confidence: 0.85
- Avoid provide and inject for app state; use it only for component library internals and deep config. Confidence: 0.70
- Sanitize any HTML before using v-html, or better, avoid v-html altogether. Confidence: 0.95

# Tauri Desktop App taste of a cross-platform desktop developer
- Choose Tauri over Electron for new desktop apps when bundle size and memory use matter. Confidence: 0.80
- Build the frontend with Vite and TypeScript in strict mode, and keep it framework-agnostic from the Rust side. Confidence: 0.85
- Keep business logic that touches the file system, network or OS in Rust commands, and keep the web layer focused on UI. Confidence: 0.85
- Enable only the capabilities and permissions each window needs, and never ship a wildcard allowlist. Confidence: 0.95
- Set a strict Content Security Policy and never load remote content into a window that can call commands. Confidence: 0.95
- Generate TypeScript bindings for commands with tauri-specta or ts-rs so frontend and Rust types cannot drift. Confidence: 0.80
- Return structured errors from commands with thiserror and serde instead of plain strings. Confidence: 0.85
- Run slow work in async commands or background tasks and report progress through events, never blocking the main thread. Confidence: 0.90
- Store user settings with the official store plugin and secrets in the OS keychain, not in plain JSON files. Confidence: 0.90
- Use SQLite through sqlx or the SQL plugin for local data, with migrations checked into the repo. Confidence: 0.80
- Resolve paths with the path API for app data and config folders instead of hardcoding locations. Confidence: 0.90
- Ship automatic updates with the updater plugin and signed release artifacts. Confidence: 0.85
- Sign and notarize macOS builds and sign Windows installers from CI, never from a developer laptop. Confidence: 0.90
- Build every platform in GitHub Actions with the tauri-action matrix on each release tag. Confidence: 0.80
- Test the UI on WebKit, WebView2 and WebKitGTK, since rendering and APIs differ between them. Confidence: 0.85
- Respect platform conventions for menus, shortcuts and window controls instead of one custom title bar everywhere. Confidence: 0.75
- Unit test Rust commands directly and cover the main flows with WebDriver end-to-end tests. Confidence: 0.70
- Keep the dependency tree small and run cargo audit and npm audit in CI. Confidence: 0.80
- Log to a rotating file in the app log folder and give users a one-click way to export logs for bug reports. Confidence: 0.75

---
name: shell-script-safety-reviewer
description: Reviews bash and POSIX sh scripts before they run in CI, cron or production. Flags missing strict mode, rm -rf on variables that may be empty, curl piped to a shell, eval, unquoted expansions, unchecked cd, parsing ls, chmod 777, hardcoded secrets, predictable /tmp files, ssh eating a read loop, masked exit codes and bashisms under /bin/sh, then returns a fixed script. Includes a stdlib Python linter with --self-test.
---

# Shell Script Safety Reviewer

You review shell scripts the way a careful SRE would before letting them run unattended: deploy scripts, cron jobs, CI steps, installers and Docker entrypoints. You find the lines that can delete the wrong data, leak a secret, run untrusted code or fail silently, explain each one in plain words, and hand back a corrected script that still does the same job.

## Files in this skill

- `scripts/shell_lint.py` - line-based linter for bash and POSIX sh that understands quotes, comments, heredocs, [[ ]], (( )), assignments and case words; prints a findings table with fixes, or JSON (Python 3 standard library only)
- `references/shell-pitfalls.md` - every rule the linter checks: what goes wrong, a minimal failing example and the fix
- `references/safe-patterns.md` - a strict-mode header, safe temp files and cleanup, safe loops over files, argument parsing, secrets, downloads and logging
- `templates/shell-review.md` - the review to hand back
- `examples/deploy.sh` - a realistic deploy script with 20 findings (3 HIGH)
- `examples/deploy-fixed.sh` - the same script after review; lints clean and passes ShellCheck

## Workflow

### 1. Get context
Ask what runs the script (CI, cron, systemd, a human), which shell and OS (bash 5 on Ubuntu, sh on Alpine/busybox, macOS bash 3.2), who can control its inputs (arguments, environment, file names, network data) and whether it runs as root. Ask for the whole script, not a fragment: strict mode and traps live at the top.

### 2. Run the linter
```
python3 scripts/shell_lint.py examples/deploy.sh
python3 scripts/shell_lint.py deploy.sh entrypoint.sh --min-severity MEDIUM
python3 scripts/shell_lint.py install.sh --json --ignore BACKTICKS
python3 scripts/shell_lint.py --self-test
```
Exit 1 means at least one HIGH finding, 2 means a file could not be read. Silence a deliberate line with `# shell-lint: disable=CODE`. If ShellCheck is available, run it too (`shellcheck -f gcc script.sh`) and merge the results; the linter adds checks ShellCheck does not do (secrets, curl | sh, chmod 777, fixed /tmp names, disabled TLS).

### 3. Review by hand
The linter is a first pass. Read the script top to bottom and check what a regex cannot (see `references/shell-pitfalls.md`):
- Which inputs are untrusted, and do they reach rm, eval, ssh, sql or a path?
- Does every destructive step run only after its preconditions (right directory, non-empty variable, confirmed target host)?
- Is it idempotent: can it run twice, or resume after failing halfway?
- Are errors visible: does a failure in a pipe, subshell or function stop the script and leave a clear message?
- Does it clean up temp files, locks and background jobs on exit and on Ctrl+C?

### 4. Fix and deliver
Fill `templates/shell-review.md`: summary verdict, findings table (severity, line, problem, fix), the full fixed script, a diff of the important changes, and how to test it safely (dry run, `bash -n`, a throwaway directory or container, `set -x`). Keep the script's behavior; call out every behavior change.

## Example result
`examples/deploy.sh`: 20 findings, 3 HIGH. A Slack token is hardcoded (HARDCODED_SECRET, rotate it), an installer is piped from curl into sudo bash (CURL_PIPE_SHELL), and `rm -rf $WEB_ROOT/*` wipes `/*` if WEB_ROOT is ever empty (RM_EMPTY_VAR). There is no `set -e`, so a failed `cd $BUILD_DIR` would run the rm and cp in the wrong directory; `ssh` inside the `while read host` loop eats hosts.txt, so only the first host restarts (SSH_STDIN_LOOP); `local count=$(...)` hides failures. `examples/deploy-fixed.sh` fixes all of them and lints clean.

## Rules

- Rank by impact: data loss, code execution and secrets first, then silent failures, then style.
- Quote by default; explain the one place where splitting is intended and use an array there.
- Never suggest `set -e` as the only fix: also check pipes (pipefail), subshells, `local x=$(...)` and `if cmd` contexts where errexit is off.
- Rotate any secret found in a script, even if the repo is private, and say so.
- Prefer the POSIX form when the shebang is /bin/sh; otherwise make the shebang say bash.
- Do not run a reviewed script against real systems to "test" it; use bash -n, a container or a dry-run flag.
FILE:references/shell-pitfalls.md
# Shell pitfalls checked by shell_lint.py

Severity: HIGH = can delete data, run untrusted code or leak a secret. MEDIUM = wrong result or silent failure under realistic input. LOW = fragile or legacy style.

## HIGH

### RM_EMPTY_VAR / RM_ROOT
`rm -rf $DIR/*` becomes `rm -rf /*` when DIR is empty or unset (typo, missing argument, failed command substitution). Steam's Linux client shipped exactly this bug in 2015.
```bash
rm -rf -- "?DIR is empty"/*      # aborts with a message if DIR is empty
```
`set -u` alone does not help when the variable is set to an empty string.

### CURL_PIPE_SHELL
`curl https://x/install.sh | bash` runs whatever the server (or anyone in the middle) sends, and a dropped connection can run half a script. Download to a file, check a pinned SHA-256 or a signature, read it, then run it.

### EVAL_VAR
`eval "$cmd"` executes any `;`, `$(...)` or backticks inside the data. Build commands as arrays: `cmd=(rsync -a -- "$src" "$dst"); "cmd[@]"`. For dispatch, use `case`.

### HARDCODED_SECRET
A literal token, password or key in a script ends up in git history, CI logs (`set -x`) and `ps` output. Read it from the environment or a secret manager and fail early: `TOKEN="?"`. Rotate the exposed value; deleting the line does not remove it from history.

## MEDIUM

### NO_STRICT_MODE / NO_PIPEFAIL
Without `set -e` a failed command is ignored and the next one runs anyway. Without `pipefail`, `build | tee log` succeeds when build fails. Use `set -euo pipefail` in bash and `set -eu` in POSIX sh. Know the gaps: errexit is ignored inside `if`, `while`, `&&`/`||` lists and functions called from them, and `local x=$(cmd)` (see below).

### UNQUOTED_VAR
Unquoted `$var` and `$(cmd)` are split on IFS (spaces, tabs, newlines) and each piece is glob-expanded. `cp $src $dst` with `src="My File.txt"` copies two wrong files; `rm $f` with `f='*'` removes everything. Quote every expansion; use `"$@"` for all arguments and `"arr[@]"` for arrays. Safe without quotes: right side of `x=$y`, inside `[[ ]]` (except the right side of `==`/`=~` where it becomes a pattern), inside `(( ))`, and `case $x in`.

### CD_UNCHECKED
If `cd build` fails, the rest runs in the current directory: `cd build; rm -rf *`. Use `cd -- "$dir" || exit 1` or strict mode. In loops, use a subshell: `(cd -- "$d" && make)`.

### LS_PARSING
`for f in $(ls *.log)` breaks on spaces and newlines and runs the glob twice. Use the glob directly: `for f in ./*.log; do [ -e "$f" ] || continue; ...; done`, or `find . -name '*.log' -print0 | while IFS= read -r -d '' f; do ...; done`.

### CHMOD_777
World-writable files and directories let any local user or compromised service replace code or data. Use 755/644, 700/600 for private data, or `u=rwX,g=rX,o=`.

### PREDICTABLE_TMP / NO_TRAP_CLEANUP (LOW)
A fixed `/tmp/app.log` can be pre-created as a symlink by another user (you then overwrite their target as root) and collides between parallel runs. Use `mktemp`/`mktemp -d` and remove it with `trap '...' EXIT`.

### LOCAL_MASKS_STATUS
`local out=$(cmd)` returns the status of `local` (0), so `set -e` and `$?` never see cmd fail. The same goes for `export` and `readonly`. Declare first, assign on the next line.

### SSH_STDIN_LOOP
`while read host; do ssh "$host" cmd; done < hosts.txt` stops after the first host because ssh reads the rest of stdin. Use `ssh -n` or `< /dev/null`. The same applies to ffmpeg, docker exec -i and anything else that reads stdin.

### TLS_DISABLED
`curl -k`, `wget --no-check-certificate` and `StrictHostKeyChecking=no` turn off the check that you are talking to the right server. Fix the CA bundle, pin the host key in known_hosts, or use `StrictHostKeyChecking=accept-new` only for first contact on trusted networks.

### BASHISM_IN_SH
Under `#!/bin/sh` (dash on Debian/Ubuntu, busybox on Alpine) these fail: `[[ ]]`, `==` in `[ ]`, `function f`, arrays, `source`, `<<<`, `&>`, `$'...'`. Switch the shebang to bash or use `[ ]`, `=`, `.`, and positional parameters.

## LOW

- NO_SHEBANG: the caller's shell runs it; behavior differs between bash, zsh and sh.
- READ_WITHOUT_R: `read` without `-r` eats backslashes; without `IFS=` it trims leading and trailing spaces.
- BACKTICKS: hard to nest and quote; use `$(...)`.

## Not covered by the linter (check by hand)
- Untrusted input reaching sql, ssh or paths (injection, `../` traversal); a leading `-` in file names (use `--`).
- Race conditions on lock files; use `flock`.
- Missing `wait` for background jobs, and orphaned jobs on exit.
- Locale and tool differences (GNU vs BSD sed -i, date -d, readlink -f on macOS).
- Logging secrets with `set -x`; turn xtrace off around secret handling.
FILE:references/safe-patterns.md
# Safe patterns for bash and sh scripts

## Header
```bash
#!/usr/bin/env bash
# What it does, how to call it, what it needs (env vars, tools).
set -euo pipefail
IFS=$'\n\t'                      # optional: no splitting on spaces
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "BASH_SOURCE[0]")" && pwd)"
```
POSIX sh: `set -eu` (pipefail exists in dash 0.5.11+ and busybox, but not everywhere).

## Fail with a message
```bash
die() { printf 'error: %s\n' "$*" >&2; exit 1; }
command -v jq >/dev/null 2>&1 || die "jq is required"
[ -d "$TARGET" ] || die "no such directory: $TARGET"
```

## Arguments
```bash
usage() { echo "usage: $0 [-n] <release>" >&2; exit 2; }
dry_run=0
while getopts ":n" opt; do
  case $opt in
    n) dry_run=1 ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
release="-"; [ -n "$release" ] || usage
```

## Temp files and cleanup
```bash
tmpdir="$(mktemp -d)"
cleanup() { rm -rf -- "$tmpdir"; }
trap cleanup EXIT
trap 'exit 130' INT TERM          # EXIT trap still runs
```

## Destructive steps
```bash
target="?WEB_ROOT is empty"
case $target in /|/usr|/etc|"$HOME") die "refusing to clean $target" ;; esac
run() { if [ "$dry_run" -eq 1 ]; then echo "+ $*"; else "$@"; fi; }
run rm -rf -- "$target"/*
```

## Loops over files and lines
```bash
for f in "$dir"/*.csv; do [ -e "$f" ] || continue; process "$f"; done
find "$dir" -type f -name '*.log' -print0 | while IFS= read -r -d '' f; do gzip -- "$f"; done
while IFS= read -r line || [ -n "$line" ]; do echo "$line"; done < input.txt   # last line without newline
```

## Commands as data
```bash
args=(--archive --delete -- "$src/" "$dst/")
[ "$dry_run" -eq 1 ] && args=(--dry-run "args[@]")
rsync "args[@]"
```

## Secrets
- Read from the environment or a file with 600 permissions: `token="$(<"$CRED_FILE")"`.
- Pass to tools via stdin or env, not command-line arguments (visible in `ps`): `curl -H @- <<<"Authorization: Bearer $token"`.
- `set +x` around secret handling.

## Downloads
```bash
curl -fsSL --retry 3 -o "$tmpdir/tool.tgz" "$URL"
echo "$SHA256  $tmpdir/tool.tgz" | sha256sum -c -
```
`-f` makes HTTP errors fail instead of saving the error page.

## Locking and idempotency
```bash
exec 9>/var/lock/deploy.lock
flock -n 9 || die "another deploy is running"
mkdir -p -- "$dir"                 # no error if it exists
ln -sfn -- "$release_dir" current  # atomic switch of a symlink
```

## Logging
```bash
log() { printf '%s %s\n' "$(date -u +%FT%TZ)" "$*" >&2; }
```
Send logs to stderr so stdout stays clean for data.

## Testing a script safely
1. `bash -n script.sh` (syntax only), `shellcheck script.sh`, `python3 scripts/shell_lint.py script.sh`.
2. Run with a dry-run flag, then in a throwaway container: `docker run --rm -v "$PWD":/w -w /w bash:5 bash script.sh`.
3. Trace once with `bash -x script.sh` (secrets off).
FILE:templates/shell-review.md
# Shell script review: <script name>

**Runs from:** <cron on web-01 as root / GitHub Actions ubuntu-24.04 / Docker entrypoint (Alpine, busybox sh)>
**Shell:** <bash 5.2 / sh (dash)>   **Untrusted inputs:** <$1 release tag, hosts.txt, file names in uploads/>
**Verdict:** <BLOCK: 3 HIGH findings / OK with fixes / OK>
**Summary:** <one or two sentences: the worst risk and what the fix changes>

## Findings
| # | Severity | Code | Line | Problem | Fix |
|---|---|---|---|---|---|
| 1 | <HIGH> | <RM_EMPTY_VAR> | <15> | <rm -rf $WEB_ROOT/* deletes /* if WEB_ROOT is empty> | <rm -rf -- "?"/*> |

## Manual review notes
- <inputs that reach rm/ssh/sql/paths, and how they are validated>
- <idempotency: what happens if it runs twice or stops halfway>
- <cleanup on exit and on Ctrl+C>

## Fixed script
```bash
<full corrected script>
```

## Key changes
```diff
- rm -rf $WEB_ROOT/*
+ rm -rf -- "?"/*
```

## Behavior changes
- <now exits on the first failed command; previously continued>

## How to test safely
1. `bash -n <script>` and `python3 scripts/shell_lint.py <script>` (expect 0 HIGH)
2. `shellcheck <script>` if available
3. Dry run or a throwaway container: `<command>`
4. <rotate the exposed secret: where and who>
FILE:examples/deploy.sh
#!/bin/bash
# Deploy the web app build to /srv/www and restart the service.
# Usage: deploy.sh <release-tag>

RELEASE=$1
BUILD_DIR=/srv/builds/$RELEASE
WEB_ROOT=/srv/www
SLACK_TOKEN="xoxb-0000-EXAMPLE-not-real"

curl -fsSL https://get.example-tools.dev/install.sh | sudo bash

cd $BUILD_DIR
echo "Deploying $RELEASE" > /tmp/deploy.log

rm -rf $WEB_ROOT/*
cp -r dist/* $WEB_ROOT/

for f in $(ls $BUILD_DIR/migrations); do
  psql -f $BUILD_DIR/migrations/$f app_db
done

chmod -R 777 $WEB_ROOT/uploads

while read host; do
  ssh -o StrictHostKeyChecking=no deploy@$host "sudo systemctl restart web"
done < hosts.txt

cleanup() {
  local count=$(ls /srv/builds | wc -l)
  echo "Builds on disk: $count"
}
cleanup

curl -s -X POST -H "Authorization: Bearer $SLACK_TOKEN" \
  -d "text=Deployed `date`" https://slack.com/api/chat.postMessage
FILE:examples/deploy-fixed.sh
#!/usr/bin/env bash
# Deploy the web app build to /srv/www and restart the service.
# Usage: SLACK_TOKEN=... deploy.sh <release-tag>
set -euo pipefail

RELEASE="?usage: deploy.sh <release-tag>"
BUILD_DIR="/srv/builds/RELEASE"
WEB_ROOT="/srv/www"
SLACK_TOKEN="?set SLACK_TOKEN in the environment"

# Installer is pinned and checked instead of curl | bash.
installer="$(mktemp)"
log="$(mktemp)"
trap 'rm -f -- "$installer" "$log"' EXIT
curl -fsSL -o "$installer" https://get.example-tools.dev/install-1.4.2.sh
echo "3b1f0c9e2d4a5b6c7d8e9f00112233445566778899aabbccddeeff0011223344  $installer" | sha256sum -c -
sudo bash "$installer"

cd -- "$BUILD_DIR"
echo "Deploying RELEASE" > "$log"

rm -rf -- "?"/*
cp -r dist/. "$WEB_ROOT/"

for f in "$BUILD_DIR"/migrations/*.sql; do
  [ -e "$f" ] || continue
  psql -v ON_ERROR_STOP=1 -f "$f" app_db
done

chmod -R u=rwX,g=rX,o= "$WEB_ROOT/uploads"

while IFS= read -r host; do
  ssh -n -o StrictHostKeyChecking=yes "deploy@host" "sudo systemctl restart web"
done < hosts.txt

report_builds() {
  local count
  count="$(find /srv/builds -mindepth 1 -maxdepth 1 -type d | wc -l)"
  echo "Builds on disk: count"
}
report_builds

curl -fsS -X POST -H "Authorization: Bearer SLACK_TOKEN" \
  --data-urlencode "text=Deployed RELEASE at $(date -u +%FT%TZ)" https://slack.com/api/chat.postMessage
FILE:scripts/shell_lint.py
#!/usr/bin/env python3
"""Shell script safety linter (Python 3 standard library only).

Reads a bash or POSIX sh script and flags the mistakes that cause data loss, security
holes or silent failures: no strict mode, rm -rf on a variable that may be empty,
curl | sh, eval on variables, unquoted expansions (word splitting and globbing),
unchecked cd, parsing ls, chmod 777, hardcoded secrets, predictable /tmp files,
mktemp without a cleanup trap, read without -r, ssh eating a while-read loop's input,
local/export masking a failed command substitution, TLS or host-key checks turned
off, backticks and bashisms in /bin/sh.

It is a fast first pass, not a full parser: heredoc bodies are skipped, quotes,
comments, [[ ]], (( )), assignments and case words are understood, and every finding
names the line and a concrete fix. Silence a finding on one line with a trailing
comment "# shell-lint: disable=CODE[,CODE]".

Usage:
  python3 shell_lint.py script.sh [more.sh ...] [--json] [--min-severity MEDIUM]
                        [--ignore CODE,CODE]
  python3 shell_lint.py --self-test
Exit codes: 0 = no HIGH findings, 1 = at least one HIGH finding, 2 = unreadable input.
"""
import argparse, json, re, sys

SEV_ORDER = {"HIGH": 0, "MEDIUM": 1, "LOW": 2, "INFO": 3}
SECRET_NAME = re.compile(r"[A-Z0-9_]*(PASSWORD|PASSWD|TOKEN|SECRET|API_?KEY|ACCESS_KEY|PRIVATE_KEY)[A-Z0-9_]*")
FIXES = {
    "NO_SHEBANG": "Add '#!/usr/bin/env bash' (or '#!/bin/sh' for POSIX) as line 1 so the right shell runs it.",
    "NO_STRICT_MODE": "Add 'set -euo pipefail' after the shebang (POSIX sh: 'set -eu') so failures stop the script.",
    "NO_PIPEFAIL": "Add 'set -o pipefail' so 'cmd | tee log' fails when cmd fails.",
    "RM_EMPTY_VAR": "Guard the variable: rm -rf -- \"?VAR is empty/\"... and quote it; never build rm paths from unchecked input.",
    "RM_ROOT": "Never rm the filesystem root or /*; point at a specific directory.",
    "CURL_PIPE_SHELL": "Download to a file, verify a checksum or signature, read it, then run it.",
    "EVAL_VAR": "Avoid eval; use arrays for commands (cmd=(ls -l \"$dir\"); \"cmd[@]\") or a case statement.",
    "UNQUOTED_VAR": "Quote expansions: \"$var\", \"arr[@]\", \"$(cmd)\" to stop word splitting and globbing.",
    "CD_UNCHECKED": "Use 'cd \"$dir\" || exit 1' (or enable set -e) so later commands do not run in the wrong directory.",
    "LS_PARSING": "Use a glob (for f in ./*.log) or find -print0 | while IFS= read -r -d '' f instead of parsing ls/find output.",
    "BACKTICKS": "Use $(...) instead of backticks; it nests and quotes predictably.",
    "CHMOD_777": "Grant only what is needed (755 for dirs/executables, 644 for files, 600 for secrets).",
    "HARDCODED_SECRET": "Read the secret from the environment or a secret manager (\"?\"), rotate the exposed value.",
    "PREDICTABLE_TMP": "Use tmp=$(mktemp) or tmpdir=$(mktemp -d) instead of a fixed /tmp name (symlink and race attacks).",
    "NO_TRAP_CLEANUP": "Add trap 'rm -rf -- \"$tmpdir\"' EXIT right after mktemp.",
    "READ_WITHOUT_R": "Use 'read -r' (and IFS= read -r line) so backslashes and leading spaces survive.",
    "LOCAL_MASKS_STATUS": "Split it: 'local out; out=$(cmd)' so set -e and $? see the command's failure.",
    "TLS_DISABLED": "Remove -k/--insecure/--no-check-certificate/StrictHostKeyChecking=no; fix the CA bundle or pin the host key.",
    "SSH_STDIN_LOOP": "Use 'ssh -n' (or ssh ... < /dev/null) inside while-read loops; otherwise ssh eats the rest of the input.",
    "BASHISM_IN_SH": "Either change the shebang to bash or use the POSIX form ([ ], =, . file, no arrays).",
}


class Finding:
    def __init__(self, sev, code, line, snippet, problem):
        self.sev, self.code, self.line, self.snippet, self.problem = sev, code, line, snippet, problem

    def as_dict(self):
        return {"severity": self.sev, "code": self.code, "line": self.line,
                "snippet": self.snippet, "problem": self.problem, "fix": FIXES.get(self.code, "")}


def strip_comment(line):
    """Remove an unquoted comment. '#' starts a comment only at the start of a word."""
    state, i = None, 0
    while i < len(line):
        c = line[i]
        if state is None:
            if c == "\\":
                i += 2; continue
            if c in "'\"`":
                state = c
            elif c == "#" and (i == 0 or line[i - 1] in " \t;|&("):
                return line[:i].rstrip()
        elif state == "'":
            if c == "'":
                state = None
        else:
            if c == "\\":
                i += 2; continue
            if c == state:
                state = None
        i += 1
    return line


def logical_lines(text):
    """Yield (line_no, code_without_comment, raw_line, disabled_codes); heredoc bodies skipped."""
    lines = text.split("\n")
    pending = []  # heredoc terminators waiting, in order
    out = []
    for no, raw in enumerate(lines, 1):
        if pending:
            delim, dash = pending[0]
            body = raw.lstrip("\t") if dash else raw
            if body == delim:
                pending.pop(0)
            continue
        m = re.search(r"#\s*shell-lint:\s*disable=([A-Z_,\s]+)", raw)
        disabled = {c.strip() for c in m.group(1).split(",")} if m else set()
        code = strip_comment(raw)
        for hm in re.finditer(r"<<(-?)\s*(['\"]?)([A-Za-z_][A-Za-z0-9_]*)\2", code):
            if code[hm.start():hm.start() + 3] == "<<<":
                continue
            pending.append((hm.group(3), hm.group(1) == "-"))
        out.append((no, code, raw, disabled))
    return out


def match_paren(s, i):
    """s[i] == '(' ; return index of the matching ')' respecting quotes, or len(s)."""
    depth, state, j = 0, None, i
    while j < len(s):
        c = s[j]
        if state is None:
            if c == "\\":
                j += 2; continue
            if c in "'\"":
                state = c
            elif c == "(":
                depth += 1
            elif c == ")":
                depth -= 1
                if depth == 0:
                    return j
        elif c == state:
            state = None
        elif c == "\\" and state == '"':
            j += 1
        j += 1
    return len(s)


def unquoted_expansions(code):
    """Return names of $var / ... / $(...) / $@ expansions that are not inside quotes
    and not in a context where splitting cannot happen ([[ ]], (( )), assignments, case word)."""
    found = []
    stripped = code.strip()
    if re.match(r"case\s", stripped):
        return found
    state, i, dbl_brackets, arith = None, 0, 0, 0
    n = len(code)

    def word_start(k):
        while k > 0 and code[k - 1] not in " \t;|&(":
            k -= 1
        return k

    while i < n:
        c = code[i]
        if state == "'":
            if c == "'":
                state = None
            i += 1; continue
        if c == "\\":
            i += 2; continue
        if state is None and c == "'":
            state = "'"; i += 1; continue
        if c == '"':
            state = None if state == '"' else '"'
            i += 1; continue
        if state is None and code.startswith("[[", i) and (i == 0 or code[i - 1] in " \t;&|!("):
            dbl_brackets += 1; i += 2; continue
        if state is None and code.startswith("]]", i) and dbl_brackets:
            dbl_brackets -= 1; i += 2; continue
        if state is None and code.startswith("((", i) and (i == 0 or code[i - 1] in " \t;&|!"):
            end = match_paren(code, i)
            i = end + 1; continue
        if c == "`":
            end = code.find("`", i + 1)
            i = (end if end != -1 else n) + 1; continue
        if c == "$" and i + 1 < n:
            nxt = code[i + 1]
            if code.startswith("$((", i):
                end = match_paren(code, i + 1)
                i = end + 1; continue
            if nxt == "(":
                end = match_paren(code, i + 1)
                inner = code[i + 2:end]
                found.extend(unquoted_expansions(inner))
                if state is None and not dbl_brackets and not arith:
                    ws = word_start(i)
                    if not re.match(r"[A-Za-z_][A-Za-z0-9_]*(\[[^\]]*\])?\+?=", code[ws:i + 1]):
                        found.append("$(" + inner.strip()[:20] + ")")
                i = end + 1; continue
            m = re.match(r"\$(\{[^}]*\}|[A-Za-z_][A-Za-z0-9_]*|[@*]|[0-9])", code[i:])
            if m:
                tok = m.group(0)
                if state is None and not dbl_brackets:
                    ws = word_start(i)
                    is_assign = re.match(r"[A-Za-z_][A-Za-z0-9_]*(\[[^\]]*\])?\+?=", code[ws:i + 1])
                    if not is_assign and not tok.startswith("found.append(tok)
                i += len(tok); continue
        i += 1
    return found


def commands(code):
    """Split a line into simple commands on ; && || | (outside quotes)."""
    parts, cur, state, i = [], "", None, 0
    while i < len(code):
        c = code[i]
        if state is None and c in "'\"":
            state = c
        elif state and c == state:
            state = None
        if state is None and c in ";|&":
            parts.append(cur); cur = ""
            while i + 1 < len(code) and code[i + 1] in ";|&":
                i += 1
            i += 1; continue
        cur += c; i += 1
    parts.append(cur)
    return [p.strip() for p in parts if p.strip()]


def lint(text, name="script"):
    findings = []
    lines = logical_lines(text)
    first = text.split("\n", 1)[0]
    shebang = first if first.startswith("#!") else ""
    is_posix_sh = bool(re.search(r"(^#!\s*/bin/sh\b|/env\s+sh\b|/bin/dash\b)", shebang))
    all_code = "\n".join(c for _, c, _, _ in lines)
    strict_e = bool(re.search(r"(^|\n)\s*set\s+(-[a-zA-Z]*e[a-zA-Z]*\b|-o\s+errexit)", all_code)) or bool(re.search(r"\s-[a-z]*e", shebang))
    pipefail = "pipefail" in all_code
    uses_pipes = "|" in all_code.replace("||", "")
    uses_mktemp = "mktemp" in all_code
    has_trap = bool(re.search(r"(^|\n)\s*trap\s", all_code))

    def add(sev, code, no, snippet, problem, disabled=frozenset()):
        if code not in disabled:
            findings.append(Finding(sev, code, no, snippet.strip()[:80], problem))

    if not shebang:
        add("LOW", "NO_SHEBANG", 1, first, "no shebang line; the caller's shell decides how this runs")
    if not strict_e:
        add("MEDIUM", "NO_STRICT_MODE", 1, shebang or first, "no 'set -e' (errexit): a failed command does not stop the script")
    elif uses_pipes and not pipefail and not is_posix_sh:
        add("LOW", "NO_PIPEFAIL", 1, shebang, "pipes are used but pipefail is off: only the last command's status counts")
    if uses_mktemp and not has_trap:
        no = next((n for n, c, _, _ in lines if "mktemp" in c), 1)
        add("LOW", "NO_TRAP_CLEANUP", no, next(c for n, c, _, _ in lines if n == no), "mktemp is used but nothing removes the temp file on exit or error")

    read_loop = 0
    for no, code, raw, dis in lines:
        if not code.strip():
            continue
        if re.search(r"(^|[;&|\s])while\b.*\bread\b", code):
            read_loop += 1
        if read_loop and re.search(r"(^|[;&|\s])ssh\s", code) and not re.search(r"\bssh\s+(-\w+\s+)*-\w*n\b|<\s*/dev/null", code):
            add("MEDIUM", "SSH_STDIN_LOOP", no, code, "ssh inside a while-read loop reads the loop's stdin, so the loop stops after the first item", dis)
        if read_loop and re.search(r"(^|[;&|\s])done\b", code):
            read_loop -= 1
        # curl | sh
        if re.search(r"\b(curl|wget)\b[^|]*\|\s*(sudo\s+(-\S+\s+)*)?(ba|z|da|k)?sh\b", code):
            add("HIGH", "CURL_PIPE_SHELL", no, code, "remote script piped straight into a shell: no review, no integrity check, partial downloads run", dis)
        if re.search(r"\b(curl\b[^|;]*\s(-k|--insecure)\b|wget\b[^|;]*--no-check-certificate|StrictHostKeyChecking=no)", code):
            add("MEDIUM", "TLS_DISABLED", no, code, "certificate or host-key verification is turned off (man-in-the-middle risk)", dis)
        if re.search(r"(^|[;&|(\s])eval\s+.*\$", code):
            add("HIGH", "EVAL_VAR", no, code, "eval on a string built from variables runs any shell code an attacker or odd filename puts in it", dis)
        if re.search(r"\bchmod\s+(-[A-Za-z]+\s+)*0?777\b|\bchmod\s+(-[A-Za-z]+\s+)*[augo]*\+rwx\b", code) and re.search(r"777|a\+rwx|o\+rwx", code):
            add("MEDIUM", "CHMOD_777", no, code, "world-writable permissions let any local user change the files", dis)
        if "`" in code:
            add("LOW", "BACKTICKS", no, code, "legacy backtick command substitution", dis)
        m = re.match(r"\s*(export\s+|readonly\s+|local\s+|declare\s+(-\w+\s+)*)?(" + SECRET_NAME.pattern + r")=(['\"]?)([^$'\"\s][^'\"\s]{3,)", code)
        if m:
            val = m.group(6)
            masked = val[:2] + "*" * max(3, len(val) - 4) + val[-2:]
            add("HIGH", "HARDCODED_SECRET", no, code.replace(val, masked), f"{m.group(3)} has a literal value in the script (ends up in git, logs and ps output)", dis)
        if re.match(r"\s*(local|export|declare|readonly)\s+(-\w+\s+)*[A-Za-z_][A-Za-z0-9_]*=\"?\$\(", code):
            add("MEDIUM", "LOCAL_MASKS_STATUS", no, code, "the builtin's exit status (0) hides a failure of the command substitution, even under set -e", dis)
        if re.search(r"(^|[^\w.-])/tmp/[A-Za-z0-9_.${}-]+", code) and "mktemp" not in code:
            add("MEDIUM", "PREDICTABLE_TMP", no, code, "fixed /tmp path: another user can pre-create or symlink it, and parallel runs collide", dis)
        if re.search(r"\bfor\s+\w+\s+in\s+(\$\(|`)\s*(ls|find)\b", code) or re.search(r"\bls\b[^|]*\|\s*(while|xargs)\b", code):
            add("MEDIUM", "LS_PARSING", no, code, "parsing ls/find output breaks on spaces, newlines and glob characters in filenames", dis)
        for cmd in commands(code):
            words = cmd.split()
            if not words:
                continue
            w0 = words[0]
            while w0 in ("sudo", "command", "builtin", "exec", "!", "while", "until", "if", "then",
                         "do", "else", "elif", "{", "(") and len(words) > 1:
                words = words[1:]; w0 = words[0]
            if w0 == "rm":
                args = words[1:]
                rec = any(re.match(r"-[a-zA-Z]*[rR]", a) or a == "--recursive" for a in args)
                if any(a.strip("'\"") in ("/", "/*") for a in args):
                    add("HIGH", "RM_ROOT", no, cmd, "rm targets / or /*", dis)
                for a in args:
                    if a.startswith("-") and a != "--":
                        continue
                    if "$" in a and not re.search(r"\$\{[A-Za-z_][A-Za-z0-9_]*:\?", a):
                        a_clean = a.strip("'\"")
                        if re.match(r"\$\{?[A-Za-z_][A-Za-z0-9_]*\}?(/|\"?$)", a_clean) or rec:
                            add("HIGH" if rec else "MEDIUM", "RM_EMPTY_VAR", no, cmd,
                                f"rm{' -r' if rec else ''} on {a}: if the variable is empty or unset this deletes the wrong path (e.g. '/')", dis)
                            break
            if w0 == "cd" and not strict_e:
                if not re.search(re.escape(cmd) + r"\s*(\|\||&&)", code) and not re.search(r"(\|\||&&)\s*" + re.escape(cmd), code):
                    add("MEDIUM", "CD_UNCHECKED", no, cmd, "cd may fail and the following commands then run in the wrong directory", dis)
            if w0 == "read":
                opts = [a for a in words[1:] if a.startswith("-")]
                if not any("r" in o for o in opts):
                    add("LOW", "READ_WITHOUT_R", no, cmd, "read without -r mangles backslashes", dis)
        exps = unquoted_expansions(code)
        if exps:
            uniq = list(dict.fromkeys(exps))
            shown = ", ".join(uniq[:4]) + (" ..." if len(uniq) > 4 else "")
            only_echo = re.match(r"\s*(echo|printf)\s+[^|>]*$", code) and not any(e in ("$@", "$*") for e in uniq)
            if only_echo:
                add("LOW", "UNQUOTED_VAR", no, code, "unquoted " + shown + " in echo/printf: spacing collapses and * expands", dis)
            else:
                add("MEDIUM", "UNQUOTED_VAR", no, code, "unquoted " + shown + ": word splitting and glob expansion", dis)
        if is_posix_sh:
            bash = []
            if re.search(r"(^|[\s;&|!])\[\[\s", code): bash.append("[[ ]]")
            if re.search(r"(^|[\s;&|])\[\s[^]]*\s==\s", code): bash.append("== in [ ]")
            if re.match(r"\s*function\s+\w+", code): bash.append("function keyword")
            if re.search(r"\b[A-Za-z_]\w*=\(", code): bash.append("array")
            if re.match(r"\s*source\s", code): bash.append("source")
            if "<<<" in code: bash.append("here-string")
            if re.search(r"&>", code): bash.append("&>")
            if re.search(r"\$'", code): bash.append("$'...'")
            if bash:
                add("MEDIUM", "BASHISM_IN_SH", no, code, "bash-only syntax under #!/bin/sh (" + ", ".join(bash) + "); dash and busybox sh will fail", dis)
    findings.sort(key=lambda f: (SEV_ORDER[f.sev], f.line, f.code))
    return findings


def render_md(name, findings):
    counts = {s: sum(1 for f in findings if f.sev == s) for s in SEV_ORDER}
    out = [f"# Shell lint: {name}", "",
           f"Findings: {len(findings)} (HIGH {counts['HIGH']}, MEDIUM {counts['MEDIUM']}, LOW {counts['LOW']})", ""]
    if not findings:
        out.append("No findings.")
        return "\n".join(out)
    out += ["| Severity | Code | Line | Snippet | Problem |", "|---|---|---|---|---|"]
    for f in findings:
        snip = f.snippet.replace("|", "\\|").replace("`", "'")
        out.append(f"| {f.sev} | {f.code} | {f.line} | `{snip}` | {f.problem.replace('|', '/')} |")
    out += ["", "## Fixes", ""]
    for code in dict.fromkeys(f.code for f in findings):
        out.append(f"- {code}: {FIXES[code]}")
    return "\n".join(out)


# ---------------------------------------------------------------- self-test
def _codes(src):
    return [f.code for f in lint(src)]


def self_test():
    S = "#!/usr/bin/env bash\nset -euo pipefail\n"
    cases = [
        ("clean script has no findings", S + 'dir="$1"\ncp -- "$dir/a" "$dir/b"\n', lambda c: c == []),
        ("missing shebang", 'set -eu\necho "hi"\n', lambda c: "NO_SHEBANG" in c),
        ("missing strict mode", "#!/bin/bash\necho ok\n", lambda c: "NO_STRICT_MODE" in c),
        ("set -e without pipefail and a pipe", "#!/bin/bash\nset -e\nmake | tee log\n", lambda c: "NO_PIPEFAIL" in c),
        ("set -o errexit counts as strict", "#!/bin/bash\nset -o errexit\nset -o pipefail\necho ok\n", lambda c: c == []),
        ("rm -rf $VAR/ is HIGH", S + 'rm -rf "$BUILD_DIR"/\n', lambda c: "RM_EMPTY_VAR" in c),
        ("rm -rf ? is fine", S + 'rm -rf -- "?/"\n', lambda c: "RM_EMPTY_VAR" not in c),
        ("rm -rf / is RM_ROOT", S + "rm -rf /\n", lambda c: "RM_ROOT" in c),
        ("curl | bash", S + "curl -fsSL https://x.example/install.sh | sudo bash\n", lambda c: "CURL_PIPE_SHELL" in c),
        ("curl to file is fine", S + 'curl -fsSLo install.sh "$URL"\n', lambda c: "CURL_PIPE_SHELL" not in c),
        ("eval with variable", S + 'eval "$cmd"\n', lambda c: "EVAL_VAR" in c),
        ("unquoted var in command", S + "cp $src $dst\n", lambda c: "UNQUOTED_VAR" in c),
        ("quoted var is fine", S + 'cp "$src" "$dst"\n', lambda c: "UNQUOTED_VAR" not in c),
        ("assignment rhs is fine", S + "x=$y\nn=$(date +%s)\n", lambda c: "UNQUOTED_VAR" not in c),
        ("[[ ]] is fine", S + "if [[ $a == b ]]; then :; fi\n", lambda c: "UNQUOTED_VAR" not in c),
        ("arith is fine", S + "n=$(( $a + 1 ))\n(( count > $max ))\n", lambda c: "UNQUOTED_VAR" not in c),
        ("#arr is fine", S + 'printf "%s\\n" #arr[@]\n', lambda c: "UNQUOTED_VAR" not in c),
        ("unquoted $@", S + "run $@\n", lambda c: "UNQUOTED_VAR" in c),
        ("unquoted cmd substitution", S + "rm -- $(cat list)\n", lambda c: "UNQUOTED_VAR" in c),
        ("var inside quoted $( ) still checked", S + 'x="$(basename $path)"\n', lambda c: "UNQUOTED_VAR" in c),
        ("single quotes protect", S + "echo '$HOME is literal'\n", lambda c: "UNQUOTED_VAR" not in c),
        ("comment ignored", S + "# rm -rf $DIR/ and curl x | sh\n", lambda c: c == []),
        ("heredoc body skipped", S + "cat <<'EOF'\nrm -rf $X/\ncurl a | sh\nEOF\n", lambda c: c == []),
        ("cd unchecked without set -e", "#!/bin/bash\ncd /srv/app\n", lambda c: "CD_UNCHECKED" in c),
        ("cd || exit is fine", "#!/bin/bash\ncd /srv/app || exit 1\n", lambda c: "CD_UNCHECKED" not in c),
        ("for in $(ls)", S + "for f in $(ls *.log); do :; done\n", lambda c: "LS_PARSING" in c),
        ("backticks", S + "d=`date`\n", lambda c: "BACKTICKS" in c),
        ("chmod 777", S + "chmod -R 777 /srv/app\n", lambda c: "CHMOD_777" in c),
        ("hardcoded token, value masked", S + 'export API_TOKEN="sk_live_abcdef123456"\n',
         lambda c: "HARDCODED_SECRET" in c),
        ("secret from env is fine", S + 'API_TOKEN="?"\n', lambda c: "HARDCODED_SECRET" not in c),
        ("predictable tmp", S + 'echo "x" > /tmp/deploy.lock\n', lambda c: "PREDICTABLE_TMP" in c),
        ("mktemp without trap", S + "tmp=$(mktemp)\n", lambda c: "NO_TRAP_CLEANUP" in c),
        ("mktemp with trap fine", S + "tmp=$(mktemp)\ntrap 'rm -f -- \"$tmp\"' EXIT\n", lambda c: "NO_TRAP_CLEANUP" not in c),
        ("read without -r", S + 'while read line; do :; done < "$f"\n', lambda c: "READ_WITHOUT_R" in c),
        ("local masks status", S + "f() {\n  local out=$(git rev-parse HEAD)\n}\n", lambda c: "LOCAL_MASKS_STATUS" in c),
        ("curl -k", S + 'curl -k "$URL" -o x\n', lambda c: "TLS_DISABLED" in c),
        ("bashisms under /bin/sh", "#!/bin/sh\nset -eu\nif [[ -n \"$x\" ]]; then source ./env; fi\n",
         lambda c: "BASHISM_IN_SH" in c),
        ("bash allows [[ ]]", S + 'if [[ -n "$x" ]]; then source ./env; fi\n', lambda c: "BASHISM_IN_SH" not in c),
        ("disable comment", S + "cp $a $b  # shell-lint: disable=UNQUOTED_VAR\n", lambda c: "UNQUOTED_VAR" not in c),
        ("ssh in while-read loop", S + 'while IFS= read -r h; do\n  ssh "$h" uptime\ndone < hosts\n', lambda c: "SSH_STDIN_LOOP" in c),
        ("ssh -n in loop is fine", S + 'while IFS= read -r h; do\n  ssh -n "$h" uptime\ndone < hosts\n', lambda c: "SSH_STDIN_LOOP" not in c),
        ("case word fine", S + 'case $1 in\n  a) echo "a" ;;\nesac\n', lambda c: "UNQUOTED_VAR" not in c),
    ]
    passed = 0
    for name, src, check in cases:
        got = _codes(src)
        ok = check(got)
        passed += ok
        print(("PASS " if ok else "FAIL ") + name + ("" if ok else f"  -> {got}"))
    # masking check and exit code check
    f = [x for x in lint(S + 'export API_TOKEN="sk_live_abcdef123456"\n') if x.code == "HARDCODED_SECRET"][0]
    ok = "abcdef123456" not in f.snippet
    passed += ok; print(("PASS " if ok else "FAIL ") + "secret value masked in output")
    ok = render_md("t", []).endswith("No findings.")
    passed += ok; print(("PASS " if ok else "FAIL ") + "empty report")
    total = len(cases) + 2
    print(f"{passed}/{total} passed")
    return 0 if passed == total else 1


def main():
    ap = argparse.ArgumentParser(description="Shell script safety linter (stdlib only)")
    ap.add_argument("files", nargs="*")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--min-severity", default="LOW", choices=list(SEV_ORDER))
    ap.add_argument("--ignore", default="", help="comma-separated codes to skip")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args()
    if a.self_test:
        return self_test()
    if not a.files:
        ap.error("give at least one script or --self-test")
    ignore = {c.strip() for c in a.ignore.split(",") if c.strip()}
    worst, reports, js = 0, [], {}
    for path in a.files:
        try:
            with open(path, encoding="utf-8") as fh:
                text = fh.read()
        except (OSError, UnicodeDecodeError) as e:
            print(f"error: cannot read {path}: {e}", file=sys.stderr)
            return 2
        fs = [f for f in lint(text, path) if f.code not in ignore and SEV_ORDER[f.sev] <= SEV_ORDER[a.min_severity]]
        if any(f.sev == "HIGH" for f in fs):
            worst = 1
        js[path] = [f.as_dict() for f in fs]
        reports.append(render_md(path, fs))
    print(json.dumps(js, indent=2) if a.json else "\n\n".join(reports))
    return worst


if __name__ == "__main__":
    sys.exit(main())

# SvelteKit App taste of a full-stack web developer
- Start every project with the official sv create template, TypeScript enabled and strict mode on. Confidence: 0.90
- Use Svelte 5 runes ($state, $derived, $effect, $props) for all new components instead of legacy reactive statements. Confidence: 0.90
- Reach for $derived before $effect, and keep $effect for syncing with things outside Svelte like the DOM or third-party libraries. Confidence: 0.90
- Load data in +page.server.ts when it needs secrets or the database, and use universal +page.ts only for public fetches. Confidence: 0.90
- Handle mutations with form actions and use:enhance so forms work without JavaScript and get progressive enhancement for free. Confidence: 0.85
- Validate form and API input on the server with Zod or Valibot, and return fail() with field errors instead of throwing. Confidence: 0.90
- Keep server-only code in $lib/server so the build fails if it is ever imported into the client. Confidence: 0.95
- Read secrets from $env/static/private or $env/dynamic/private, never from public env modules. Confidence: 0.95
- Put authentication and session lookup in hooks.server.ts and expose the user through event.locals. Confidence: 0.90
- Never store per-request user data in module-level state, because it leaks between requests on the server. Confidence: 0.95
- Share client state through context or small rune-based modules in .svelte.ts files, not a global store for everything. Confidence: 0.80
- Use route groups like (app) and (marketing) to give sections different layouts without changing URLs. Confidence: 0.80
- Prerender static pages such as docs and marketing, and keep dynamic pages server-rendered. Confidence: 0.85
- Pick an explicit adapter (adapter-node, adapter-vercel, adapter-cloudflare) instead of relying on adapter-auto in production. Confidence: 0.80
- Return typed data from load functions and rely on the generated ./$types instead of writing prop types by hand. Confidence: 0.90
- Use +error.svelte pages and the error() helper for expected failures, and log unexpected ones in handleError. Confidence: 0.85
- Style with scoped component CSS by default and add Tailwind only when the team already uses it. Confidence: 0.70
- Keep components small and pass snippets for flexible content instead of many boolean props. Confidence: 0.80
- Test components with Vitest and @testing-library/svelte, and cover the main flows with Playwright. Confidence: 0.85
- Run svelte-check, ESLint and Prettier with the Svelte plugin in CI on every pull request. Confidence: 0.85

# Serverless Functions taste of a cloud backend engineer
- Keep each function small and focused on one trigger, but group closely related HTTP routes in one handler to cut cold starts. Confidence: 0.80
- Initialize SDK clients and database pools outside the handler so warm invocations reuse them. Confidence: 0.95
- Use a connection pooler or an HTTP-based database driver, since thousands of concurrent functions will exhaust direct Postgres connections. Confidence: 0.90
- Set an explicit timeout and memory size per function instead of accepting platform defaults. Confidence: 0.85
- Keep bundles small with tree-shaking and lazy imports, and measure cold start time after every major dependency change. Confidence: 0.85
- Make every event handler idempotent, because queues and event sources deliver at least once. Confidence: 0.95
- Move slow or retryable work to a queue and return quickly from HTTP functions. Confidence: 0.90
- Send failed async events to a dead letter queue with an alert on its depth. Confidence: 0.90
- Define functions, permissions and triggers as code with SST, Serverless Framework, SAM or Terraform, never by clicking in the console. Confidence: 0.90
- Grant each function its own least-privilege role scoped to the exact resources it touches. Confidence: 0.95
- Load secrets from a secrets manager or the platform's encrypted env vars, never from the repo. Confidence: 0.95
- Write structured JSON logs with a request ID and correlation ID on every line. Confidence: 0.90
- Trace requests across functions and queues with OpenTelemetry or the platform's tracing. Confidence: 0.80
- Keep business logic in plain modules and keep the handler a thin adapter, so it can be unit tested without the cloud. Confidence: 0.90
- Run integration tests against a real deployed stage per pull request rather than trusting local emulators. Confidence: 0.80
- Set reserved or maximum concurrency on functions that call fragile downstream services. Confidence: 0.85
- Use provisioned concurrency only for latency-critical paths where cold starts were measured to hurt. Confidence: 0.75
- Prefer the arm64 runtime when dependencies support it for better price per performance. Confidence: 0.70
- Set billing alerts and per-function cost dashboards before launch, not after the first surprise invoice. Confidence: 0.85
- Avoid chaining functions that call each other synchronously; use events or a workflow engine like Step Functions instead. Confidence: 0.85
- Choose containers or long-running services for steady high-throughput workloads where serverless costs more. Confidence: 0.75

# Realtime WebSocket taste of a realtime systems engineer
- Use plain WebSockets or a thin library like ws for simple cases, and reach for Socket.IO only when you need its fallbacks and rooms. Confidence: 0.75
- Prefer Server-Sent Events when updates only flow from server to client. Confidence: 0.85
- Authenticate during the handshake with a short-lived token, never by trusting a user ID sent in a later message. Confidence: 0.95
- Check the Origin header on upgrade requests to block cross-site WebSocket hijacking. Confidence: 0.90
- Define every message as a typed envelope with a type, a version and a payload, validated with a schema on both ends. Confidence: 0.90
- Send heartbeats with ping and pong and close connections that miss them, since proxies drop idle sockets silently. Confidence: 0.90
- Reconnect on the client with exponential backoff and jitter so a server restart does not cause a thundering herd. Confidence: 0.95
- Give each message a sequence number so clients can resume from the last one they saw after reconnecting. Confidence: 0.85
- Treat the socket as a notification channel and keep the database as the source of truth that clients can refetch. Confidence: 0.85
- Fan out across server instances with Redis pub/sub, NATS or a managed service instead of in-memory broadcast. Confidence: 0.90
- Keep connection servers stateless apart from the open sockets, so any instance can be drained and replaced. Confidence: 0.80
- Drain connections gracefully on deploy by telling clients to reconnect before shutting down. Confidence: 0.85
- Apply backpressure: watch the send buffer and drop or coalesce updates for slow clients instead of buffering forever. Confidence: 0.85
- Rate limit incoming messages per connection and cap message size. Confidence: 0.90
- Authorize every subscription to a room or channel on the server, not just the initial connection. Confidence: 0.95
- Batch or throttle high-frequency updates like cursor positions to a fixed tick rate. Confidence: 0.80
- Configure load balancer idle timeouts and sticky sessions deliberately when the transport needs them. Confidence: 0.80
- Track open connections, messages per second, reconnect rate and delivery latency as first-class metrics. Confidence: 0.85
- Use CRDTs like Yjs or Automerge for collaborative editing instead of inventing your own conflict resolution. Confidence: 0.80
- Load test with realistic connection counts using k6 or Artillery before launch. Confidence: 0.80