Backend Prompts

# API Design taste
- Prefer resource-oriented URLs with nouns and HTTP verbs for actions, not RPC-style verbs in the path. Confidence: 0.95
- Version APIs in the URL path (`/v1/...`) and keep breaking changes out of an existing major version. Confidence: 0.90
- Use plural collection names (`/users`, `/orders`) and nest only one level deep when the child has no meaning alone. Confidence: 0.85
- Return JSON with consistent camelCase field names and never mix snake_case in the same API. Confidence: 0.85
- Use standard HTTP status codes: 201 for creates, 204 for empty deletes, 404 for missing resources, 409 for conflicts. Confidence: 0.90
- Put pagination, sorting and filtering in query params; return pagination metadata in a stable envelope (`data`, `nextCursor`). Confidence: 0.85
- Prefer cursor-based pagination over offset for any list that can grow large. Confidence: 0.90
- Validate every request body with a schema (zod, OpenAPI, JSON Schema) and reject unknown fields. Confidence: 0.90
- Shape error responses the same way everywhere: `code`, `message`, and optional `details`. Confidence: 0.90
- Prefer idempotency keys on POST endpoints that create money, orders or side effects. Confidence: 0.85
- Use ETags or `If-Match` for concurrent updates on contested resources. Confidence: 0.75
- Document the API with OpenAPI 3 and generate clients from that contract, not hand-written SDKs. Confidence: 0.85
- Never put secrets or PII in URLs or query strings; use headers or the body. Confidence: 0.95
- Prefer explicit expansion (`?include=`) over always embedding related entities. Confidence: 0.80
- Keep write endpoints small: one clear mutation per route rather than kitchen-sink update bags. Confidence: 0.80
- Rate-limit by API key or user and return `Retry-After` when limited. Confidence: 0.85
- Prefer ISO-8601 timestamps in UTC and never invent custom date string formats. Confidence: 0.95

# Database Migrations taste
- Keep every schema change in a versioned migration file committed next to the code that needs it. Confidence: 0.95
- Write migrations in plain SQL rather than ORM-generated diffs you have not read. Confidence: 0.80
- Never edit a migration that has already run in a shared environment; add a new one instead. Confidence: 0.95
- Use the expand and contract pattern: add the new column, backfill, switch reads, then drop the old column in a later release. Confidence: 0.90
- Make every migration safe to run while the previous app version is still serving traffic. Confidence: 0.90
- Add new columns as nullable or with a constant default to avoid long table rewrites. Confidence: 0.85
- Create indexes on large Postgres tables with CREATE INDEX CONCURRENTLY outside a transaction. Confidence: 0.90
- Set a short lock_timeout and statement_timeout at the top of each migration. Confidence: 0.85
- Run large backfills as batched background jobs, not inside the migration itself. Confidence: 0.90
- Keep one logical change per migration so failures are easy to read and retry. Confidence: 0.80
- Name migration files with a UTC timestamp prefix and a short verb phrase like 20261017_add_orders_status. Confidence: 0.75
- Prefer forward-only migrations and fix mistakes with a new migration instead of relying on down scripts. Confidence: 0.70
- Run migrations as a separate deploy step before the new application code starts. Confidence: 0.85
- Test every migration in CI against a fresh database and against a copy of production-sized data. Confidence: 0.80
- Add foreign keys and check constraints as NOT VALID first, then validate them in a separate step. Confidence: 0.80
- Rename columns and tables only through a view or dual-write period, never in a single step. Confidence: 0.75
- Store the applied migration history in the database and fail the deploy if it has drifted. Confidence: 0.85
- Review migrations as carefully as application code, with the generated SQL visible in the pull request. Confidence: 0.90
- Keep seed and fixture data out of schema migrations. Confidence: 0.80

# Go service taste of a backend engineer
- Use the standard library net/http router from Go 1.22 before reaching for a framework. Confidence: 0.85
- Lay the project out as cmd/<service>/main.go plus internal/ packages, and skip pkg/. Confidence: 0.80
- Keep main.go thin: read config, wire dependencies and start the server. Confidence: 0.90
- Pass context.Context as the first argument to every function that does I/O. Confidence: 0.95
- Wrap errors with fmt.Errorf and %w, and check them with errors.Is and errors.As. Confidence: 0.95
- Log with log/slog in JSON format and include a request ID on every line. Confidence: 0.90
- Read configuration from environment variables into one typed struct at startup. Confidence: 0.85
- Fail fast at startup when required configuration is missing. Confidence: 0.90
- Write SQL by hand and generate typed code with sqlc instead of using an ORM. Confidence: 0.80
- Use pgx as the Postgres driver with a connection pool. Confidence: 0.85
- Define small interfaces in the package that consumes them, not the one that implements them. Confidence: 0.85
- Use table-driven tests with t.Run subtests. Confidence: 0.90
- Run go test with -race in CI on every pull request. Confidence: 0.90
- Lint with golangci-lint using a short, curated list of linters. Confidence: 0.80
- Shut down gracefully on SIGTERM with signal.NotifyContext and server.Shutdown. Confidence: 0.90
- Set explicit read, write and idle timeouts on every http.Server and http.Client. Confidence: 0.90
- Expose /healthz for liveness and /readyz for readiness. Confidence: 0.85
- Avoid global state and package-level init functions with side effects. Confidence: 0.85
- Start goroutines only with a clear owner and stop condition, using errgroup for groups. Confidence: 0.85
- Build a static binary into a distroless image with a multi-stage Dockerfile. Confidence: 0.80
- Add generics only when they remove real duplication. Confidence: 0.70

---
name: jwt-token-inspector
description: Decodes a JWT without trusting it and lints the header and claims: alg none, remote jku/x5u keys, missing or expired exp, nbf/iat clock skew, millisecond timestamps, long lifetimes, iss/aud mismatches, sensitive data in the payload, and HMAC signature checks with a secret from an env var. Use when a login or API call fails with 401, or when reviewing token settings. Includes a stdlib Python inspector with --self-test.
---

# JWT Token Inspector

You help developers understand and debug JSON Web Tokens safely. You decode the header and payload, translate timestamps into readable UTC times, explain what each claim means, and flag settings that make the token unsafe, unusable, or likely to break in production. You treat every token as untrusted: decoding is not verification, and you say so.

## Files in this skill

- `scripts/jwt_inspect.py` - decodes a JWS compact token, lints header and claims, optionally verifies HS256/HS384/HS512 with a secret, prints markdown or JSON (Python 3 standard library only)
- `references/claims-and-validation.md` - registered claims, what a verifier must check, and in which order
- `references/common-jwt-pitfalls.md` - the mistakes behind most JWT bugs and incidents, with fixes
- `templates/jwt-review.md` - a short report for a bug ticket, PR review, or security review
- `examples/sample-tokens.md` - three demo tokens (long-lived, alg none, remote jku with millisecond exp) and expected findings

## Workflow

### 1. Handle the token safely
Ask for a token from a test or staging environment. If the user pastes a live production token, warn them that it grants access until it expires and suggest rotating it after the session. Never repeat the full token or signature back in your answer.

### 2. Run the inspector
```
python3 scripts/jwt_inspect.py "<token>"
python3 scripts/jwt_inspect.py "<token>" --now 2026-10-19T09:00:00Z --leeway 60
python3 scripts/jwt_inspect.py "<token>" --expect-iss https://auth.example.com --expect-aud api://orders
JWT_SECRET=... python3 scripts/jwt_inspect.py "<token>" --secret-env JWT_SECRET
python3 scripts/jwt_inspect.py --self-test
```
A `Bearer ` prefix is stripped. Use `-` to read the token from stdin. Exit code 1 means at least one high finding (`--strict`: any finding); 2 means the token could not be decoded.

### 3. Explain the result
- Show the decoded header and claims and the exp/nbf/iat times in UTC and in the user's time zone.
- Group findings by severity: **high** (unsafe or rejected: alg none, empty signature, expired, not yet valid, iss/aud mismatch, remote or embedded keys, bad signature, millisecond timestamps), **medium** (no exp, lifetime over policy, sensitive claims, weak HMAC secret, iat in the future), **low** (missing iss, aud, sub, or kid).
- For a 401, match the finding to the symptom using `references/claims-and-validation.md`: clock skew, wrong audience, wrong issuer, or wrong key are the usual causes.

### 4. Recommend fixes
Use `references/common-jwt-pitfalls.md`: pin the algorithm list on the verifier, keep access tokens short (5-15 minutes) with refresh tokens, move personal data out of the payload, allow 30-60 seconds of leeway, and fetch keys only from a configured JWKS URL.

### 5. Report
Fill `templates/jwt-review.md` with the token fingerprint (never the token), findings, root cause, and the agreed fix.

## Rules

- Decoding is not verification. Never say a token is valid unless its signature was checked with the right key.
- Never print or store full tokens, signatures, or secrets. Pass secrets through an environment variable, not the command line.
- Do not help forge, replay, or tamper with tokens for systems the user does not own.
- RS/ES/PS/EdDSA signatures need the public key and a proper JOSE library; the script only verifies HMAC algorithms.
FILE:references/claims-and-validation.md
# Claims and Validation Order

## Registered claims (RFC 7519)
| Claim | Meaning | Type | Notes |
|---|---|---|---|
| iss | Issuer | string/URL | Pin it on the verifier; one issuer per trust domain |
| sub | Subject (user or service id) | string | Stable id, not an email that can change |
| aud | Audience | string or array | The API must find its own id in it |
| exp | Expiration time | NumericDate (seconds) | Required for access tokens |
| nbf | Not before | NumericDate | Token is rejected until then |
| iat | Issued at | NumericDate | Used for lifetime and freshness checks |
| jti | Token id | string | Enables revocation lists and replay detection |

NumericDate is seconds since 1970-01-01T00:00:00Z. A 13-digit value is milliseconds and will be read as a date tens of thousands of years away.

## Header parameters
- `alg` - signing algorithm. The verifier decides which values are allowed; the token never does.
- `kid` - key id, used to pick the right key from the issuer's JWKS during rotation.
- `typ` - usually `JWT` or `at+jwt` (access token profile, RFC 9068).
- `jku`, `x5u`, `jwk` - point to or embed keys. Ignore them unless they match a pinned, trusted source.

## What a verifier must check, in order
1. Token has three segments and decodes to JSON objects.
2. `alg` is in the verifier's allow-list (for example only RS256). Reject `none`.
3. Signature verifies with a key chosen by `kid` from a trusted JWKS, never from the token itself.
4. `exp` is in the future and `nbf` is in the past, both with small leeway (30-60 s).
5. `iss` equals the expected issuer.
6. `aud` contains this service's id.
7. Application rules: scopes, roles, tenant, `jti` not revoked.

## Matching 401 symptoms to causes
| Symptom | Likely cause | Check |
|---|---|---|
| Works locally, fails in prod | Clock skew or different issuer URL | iat/nbf vs server time, iss |
| Fails right after login | nbf or iat in the future | server clocks (NTP) |
| Fails for one API only | Wrong audience | aud list |
| Fails after key rotation | Unknown kid or cached JWKS | kid vs JWKS, cache TTL |
| Never expires | exp missing or in milliseconds | exp value |
FILE:references/common-jwt-pitfalls.md
# Common JWT Pitfalls

1. **Accepting `alg: none`.** Unsigned tokens are forgeable. Fix: pass an explicit algorithm allow-list to the verify call.
2. **Algorithm confusion (RS256 vs HS256).** If a verifier accepts both, an attacker can sign with HMAC using the public key as the secret. Fix: one algorithm family per key.
3. **Trusting keys from the token.** `jku`, `x5u`, or an embedded `jwk` let the sender choose the key. Fix: load keys only from a configured JWKS URL.
4. **Weak HMAC secrets.** Short or dictionary secrets can be brute-forced offline from one token. Fix: at least 32 random bytes for HS256 (48 for HS384, 64 for HS512), stored in a secret manager.
5. **Long-lived access tokens.** A leaked token works until exp. Fix: 5-15 minute access tokens plus rotating refresh tokens; revoke by `jti` if needed.
6. **Personal data in the payload.** Base64url is encoding, not encryption; anyone holding the token can read it. Fix: keep ids and scopes only, or use JWE.
7. **Milliseconds instead of seconds.** Common with `Date.now()` in JavaScript. Fix: `Math.floor(Date.now() / 1000)`.
8. **No leeway.** Small clock differences cause random 401s. Fix: 30-60 seconds of leeway and NTP on all hosts.
9. **Skipping aud/iss checks.** A token for one service gets replayed against another. Fix: always validate both.
10. **Tokens in URLs or logs.** They leak through referrers, proxies, and log search. Fix: send in the Authorization header and redact in logs.
FILE:templates/jwt-review.md
# JWT Review - <service or ticket>

**Token fingerprint:** sha256 <first 12 hex chars>  **Environment:** <dev / staging / prod>  **Checked at:** <UTC time>
**Issuer:** <iss>  **Audience:** <aud>  **Algorithm:** <alg>  **Lifetime:** <exp - iat>

## Findings
| Severity | Code | Detail | Fix |
|---|---|---|---|
| | | | |

## Root cause
<one or two sentences tying the finding to the reported symptom>

## Agreed changes
- [ ] Verifier: <allow-list, leeway, iss/aud checks>
- [ ] Issuer: <lifetime, claims, key size, kid>
- [ ] Rotate exposed token or secret: <yes/no, owner>

## Notes
Decoded with scripts/jwt_inspect.py. The token itself is not stored in this report.
FILE:examples/sample-tokens.md
# Sample Tokens (demo only, no real systems)

Run each with:
```
python3 scripts/jwt_inspect.py "<token>" --now 2026-10-19T09:00:00Z --expect-aud api://orders
```

## 1. Long-lived customer token (HS256, demo secret "dev-secret")
```
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2F1dGguc2hvcGZyb250LmV4YW1wbGUiLCJzdWIiOiJjdXN0Xzg4NDEiLCJhdWQiOlsiYXBpOi8vb3JkZXJzIiwiYXBpOi8vcHJvZmlsZSJdLCJpYXQiOjE3OTIzOTUwMDAsImV4cCI6MTc5Mjk5OTgwMCwiZW1haWwiOiJtaW5hQGV4YW1wbGUuY29tIiwicm9sZXMiOlsiY3VzdG9tZXIiXSwiY2FyZF9sYXN0NCI6IjQyNDIifQ.xupeoN2LYDYe-FKD_4qOh7KCw6G68Zi1pCms7dtUGh0
```
Expected: exit 0; medium lifetime_long (7.0d over 1.0d policy), medium sensitive_claim (card_last4). With `--secret-env` set to "dev-secret": signature verified, plus medium weak_secret (10 bytes).

## 2. Unsigned admin token (alg none)
```
eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJpc3MiOiJodHRwczovL2F1dGguc2hvcGZyb250LmV4YW1wbGUiLCJzdWIiOiJhZG1pbiIsInJvbGUiOiJhZG1pbiIsImlhdCI6MTc5MjM5NTAwMH0.
```
Expected: exit 1; high alg_none, high aud_mismatch (no aud but api://orders required), medium exp_missing.

## 3. Remote key URL and millisecond expiry (RS256)
```
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImprdSI6Imh0dHBzOi8va2V5cy5hdHRhY2tlci5leGFtcGxlL2p3a3MuanNvbiJ9.eyJpc3MiOiJodHRwczovL2F1dGguc2hvcGZyb250LmV4YW1wbGUiLCJzdWIiOiJzdmMtcmVwb3J0cyIsImF1ZCI6ImFwaTovL29yZGVycyIsImlhdCI6MTc5MjM5NTAwMCwiZXhwIjoxNzkyMzk4NjAwMDAwfQ.bm90LWEtcmVhbC1zaWduYXR1cmUtanVzdC1hLWRlbW8
```
Expected: exit 1; high exp_milliseconds, high header_jku, low kid_missing.
FILE:scripts/jwt_inspect.py
#!/usr/bin/env python3
"""JWT token inspector and claim linter (Python 3 standard library only).

Decodes a JSON Web Token (JWS compact form: header.payload.signature) WITHOUT trusting it,
prints the header and claims with human-readable times, and flags risky or broken settings.
Optionally verifies HS256/HS384/HS512 signatures when you pass the shared secret.

Usage:
  python3 jwt_inspect.py <token>                     # or "-" to read the token from stdin
  python3 jwt_inspect.py <token> --now 2026-10-19T09:00:00Z --leeway 60
  python3 jwt_inspect.py <token> --expect-iss https://auth.example.com --expect-aud api://orders
  python3 jwt_inspect.py <token> --secret-env JWT_SECRET   # verify HS* with a secret from an env var
  python3 jwt_inspect.py <token> --max-lifetime 3600 --json
  python3 jwt_inspect.py --self-test

Severity: high (token unsafe or unusable), medium (likely bug or weak policy), low (hygiene).
Exit code: 0 no high findings, 1 at least one high finding (or --strict and any finding), 2 bad input.
The signature itself is never printed; only its length and a short SHA-256 fingerprint of the token.
"""
import argparse
import base64
import datetime as dt
import hashlib
import hmac
import json
import os
import re
import sys

HS_ALGS = {"HS256": hashlib.sha256, "HS384": hashlib.sha384, "HS512": hashlib.sha512}
MIN_HS_SECRET = {"HS256": 32, "HS384": 48, "HS512": 64}
KNOWN_ALGS = set(HS_ALGS) | {"RS256", "RS384", "RS512", "PS256", "PS384", "PS512",
                             "ES256", "ES384", "ES512", "EdDSA"}
SENSITIVE = re.compile(r"(pass(word)?|pwd|secret|ssn|social|credit|card|cvv|iban|private|api_?key|refresh_?token)", re.I)
TIME_CLAIMS = ("exp", "nbf", "iat")


class TokenError(ValueError):
    pass


def b64url_decode(part, what):
    if not re.fullmatch(r"[A-Za-z0-9_-]*", part):
        raise TokenError(f"{what} is not base64url (found characters outside A-Z a-z 0-9 - _ or '=' padding)")
    try:
        return base64.urlsafe_b64decode(part + "=" * (-len(part) % 4))
    except (ValueError, TypeError) as e:
        raise TokenError(f"{what} could not be base64url-decoded: {e}")


def b64url_encode(raw):
    return base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii")


def decode(token):
    token = token.strip()
    if token.lower().startswith("bearer "):
        token = token[7:].strip()
    parts = token.split(".")
    if len(parts) == 5:
        raise TokenError("5 segments: this is an encrypted JWE, not a signed JWS; it cannot be inspected without the key")
    if len(parts) != 3:
        raise TokenError(f"expected 3 dot-separated segments, found {len(parts)}")
    out = {}
    for name, part in (("header", parts[0]), ("payload", parts[1])):
        raw = b64url_decode(part, name)
        try:
            obj = json.loads(raw.decode("utf-8"))
        except (UnicodeDecodeError, ValueError):
            raise TokenError(f"{name} is not valid UTF-8 JSON")
        if not isinstance(obj, dict):
            raise TokenError(f"{name} must be a JSON object")
        out[name] = obj
    out["signature"] = b64url_decode(parts[2], "signature")
    out["signing_input"] = (parts[0] + "." + parts[1]).encode("ascii")
    out["fingerprint"] = hashlib.sha256(token.encode("utf-8")).hexdigest()[:12]
    return out


def parse_now(value):
    if value is None:
        return int(dt.datetime.now(dt.timezone.utc).timestamp())
    if re.fullmatch(r"\d+", value):
        return int(value)
    t = dt.datetime.fromisoformat(value.replace("Z", "+00:00"))
    if t.tzinfo is None:
        t = t.replace(tzinfo=dt.timezone.utc)
    return int(t.timestamp())


def iso(ts):
    try:
        return dt.datetime.fromtimestamp(ts, dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
    except (OverflowError, OSError, ValueError, TypeError):
        return "invalid"


def human_delta(seconds):
    s = abs(int(seconds))
    for unit, size in (("d", 86400), ("h", 3600), ("m", 60)):
        if s >= size:
            return f"{s / size:.1f}{unit}"
    return f"{s}s"


def verify_hs(tok, secret):
    alg = tok["header"].get("alg")
    fn = HS_ALGS.get(alg)
    if fn is None:
        return None
    mac = hmac.new(secret.encode("utf-8"), tok["signing_input"], fn).digest()
    return hmac.compare_digest(mac, tok["signature"])


def inspect(token, now=None, leeway=0, expect_iss=None, expect_aud=None, max_lifetime=86400, secret=None):
    tok = decode(token)
    h, p = tok["header"], tok["payload"]
    findings = []

    def add(sev, code, msg):
        findings.append({"severity": sev, "code": code, "message": msg})

    alg = h.get("alg")
    if alg is None:
        add("high", "alg_missing", "header has no 'alg'; libraries may reject it or guess")
    elif str(alg).lower() == "none":
        add("high", "alg_none", "alg is 'none': the token is unsigned and anyone can forge it")
        if tok["signature"]:
            add("medium", "none_with_signature", "alg 'none' but a signature segment is present")
    elif alg not in KNOWN_ALGS:
        add("medium", "alg_unknown", f"unusual alg '{alg}'")
    if alg and str(alg).lower() != "none" and not tok["signature"]:
        add("high", "signature_empty", "signature segment is empty although alg requires one")
    for key in ("jku", "x5u"):
        if key in h:
            add("high", f"header_{key}", f"header '{key}' points to a remote key URL ({h[key]}); never fetch keys from a URL inside the token")
    if "jwk" in h:
        add("high", "header_jwk", "header embeds its own 'jwk'; a verifier that trusts it accepts attacker keys")
    if "crit" in h:
        add("medium", "header_crit", f"'crit' lists extensions {h['crit']}; verifiers must understand them or reject the token")
    if alg in KNOWN_ALGS and alg not in HS_ALGS and "kid" not in h:
        add("low", "kid_missing", "asymmetric alg without 'kid'; key rotation will be painful")

    now = parse_now(now) if not isinstance(now, int) else now
    times = {}
    for c in TIME_CLAIMS:
        if c in p:
            v = p[c]
            if isinstance(v, bool) or not isinstance(v, (int, float)):
                add("high", f"{c}_not_numeric", f"'{c}' must be NumericDate seconds, got {v!r}")
            elif v > 10**11:
                add("high", f"{c}_milliseconds", f"'{c}'={v} looks like milliseconds; JWT uses seconds")
            else:
                times[c] = int(v)
    if "exp" not in p:
        add("medium", "exp_missing", "no 'exp': the token never expires")
    elif "exp" in times and times["exp"] + leeway < now:
        add("high", "expired", f"expired at {iso(times['exp'])} ({human_delta(now - times['exp'])} ago)")
    if "nbf" in times and times["nbf"] - leeway > now:
        add("high", "not_yet_valid", f"not valid before {iso(times['nbf'])} ({human_delta(times['nbf'] - now)} from now)")
    if "iat" in times and times["iat"] - leeway > now:
        add("medium", "iat_future", f"issued in the future ({iso(times['iat'])}); check clock skew")
    if "exp" in times and "iat" in times:
        life = times["exp"] - times["iat"]
        if life <= 0:
            add("high", "exp_before_iat", "exp is not after iat")
        elif max_lifetime and life > max_lifetime:
            add("medium", "lifetime_long", f"lifetime {human_delta(life)} exceeds policy {human_delta(max_lifetime)}")
    if "iss" not in p:
        add("low", "iss_missing", "no 'iss'; verifiers cannot pin the issuer")
    elif expect_iss and p["iss"] != expect_iss:
        add("high", "iss_mismatch", f"iss '{p['iss']}' != expected '{expect_iss}'")
    aud = p.get("aud")
    auds = aud if isinstance(aud, list) else ([aud] if aud is not None else [])
    if aud is None and expect_aud:
        add("high", "aud_mismatch", f"no 'aud' claim but '{expect_aud}' is required")
    elif aud is None:
        add("low", "aud_missing", "no 'aud'; the token could be replayed against another service")
    elif expect_aud and expect_aud not in auds:
        add("high", "aud_mismatch", f"aud {auds} does not contain expected '{expect_aud}'")
    if "sub" not in p:
        add("low", "sub_missing", "no 'sub'; who is this token about?")
    for k in p:
        if SENSITIVE.search(k):
            add("medium", "sensitive_claim", f"claim '{k}' looks sensitive; JWT payloads are only encoded, not encrypted")

    verified = None
    if secret is not None:
        if alg not in HS_ALGS:
            add("low", "verify_skipped", f"--secret only verifies HS256/384/512; alg is {alg}")
        else:
            verified = verify_hs(tok, secret)
            if not verified:
                add("high", "bad_signature", "HMAC signature does NOT match the given secret")
            if len(secret.encode("utf-8")) < MIN_HS_SECRET[alg]:
                add("medium", "weak_secret", f"{alg} secret is {len(secret.encode('utf-8'))} bytes; use at least {MIN_HS_SECRET[alg]} random bytes")

    order = {"high": 0, "medium": 1, "low": 2}
    findings.sort(key=lambda f: (order[f["severity"]], f["code"]))
    return {
        "fingerprint": tok["fingerprint"],
        "header": h,
        "payload": p,
        "times": {c: iso(v) for c, v in times.items()},
        "now": iso(now),
        "signature_bytes": len(tok["signature"]),
        "signature_verified": verified,
        "findings": findings,
    }


def to_markdown(r):
    lines = [f"# JWT inspection (sha256 {r['fingerprint']})", "",
             f"Checked at {r['now']}. Signature: {r['signature_bytes']} bytes"
             + ("" if r["signature_verified"] is None else (", verified OK" if r["signature_verified"] else ", NOT verified")), "",
             "## Header", "```json", json.dumps(r["header"], indent=2, ensure_ascii=False), "```", "",
             "## Claims", "```json", json.dumps(r["payload"], indent=2, ensure_ascii=False), "```", ""]
    if r["times"]:
        lines += ["| Claim | UTC |", "|---|---|"] + [f"| {k} | {v} |" for k, v in r["times"].items()] + [""]
    lines += ["## Findings"]
    if not r["findings"]:
        lines.append("No findings.")
    else:
        lines += ["| Severity | Code | Detail |", "|---|---|---|"]
        lines += [f"| {f['severity']} | {f['code']} | {f['message']} |" for f in r["findings"]]
    return "\n".join(lines)


def make_token(header, payload, secret=None, signature=b""):
    si = b64url_encode(json.dumps(header, separators=(",", ":")).encode()) + "." + \
         b64url_encode(json.dumps(payload, separators=(",", ":")).encode())
    if secret is not None:
        signature = hmac.new(secret.encode(), si.encode(), HS_ALGS[header["alg"]]).digest()
    return si + "." + b64url_encode(signature)


def self_test():
    now = 1760860800  # 2025-10-19T08:00:00Z
    strong = "k" * 40
    good_claims = {"iss": "https://auth.example.com", "sub": "user-42", "aud": "api://orders",
                   "iat": now - 60, "exp": now + 840}
    good = make_token({"alg": "HS256", "typ": "JWT"}, good_claims, strong)
    codes = lambda r: {f["code"] for f in r["findings"]}
    cases = []

    def check(name, cond):
        cases.append((name, bool(cond)))

    r = inspect(good, now=now, secret=strong, expect_iss="https://auth.example.com", expect_aud="api://orders")
    check("clean token has no findings", r["findings"] == [])
    check("clean token signature verified", r["signature_verified"] is True)
    check("bearer prefix accepted", inspect("Bearer " + good, now=now)["payload"]["sub"] == "user-42")
    check("wrong secret flagged", "bad_signature" in codes(inspect(good, now=now, secret="x" * 40)))
    check("weak secret flagged", "weak_secret" in codes(inspect(make_token({"alg": "HS256"}, good_claims, "short"), now=now, secret="short")))
    check("alg none flagged high", "alg_none" in codes(inspect(make_token({"alg": "none"}, good_claims), now=now)))
    check("expired flagged", "expired" in codes(inspect(good, now=now + 3600)))
    check("leeway tolerates small skew", "expired" not in codes(inspect(good, now=now + 850, leeway=30)))
    check("nbf in future flagged", "not_yet_valid" in codes(inspect(make_token({"alg": "HS256"}, dict(good_claims, nbf=now + 600), strong), now=now)))
    check("exp missing flagged", "exp_missing" in codes(inspect(make_token({"alg": "HS256"}, {"sub": "a", "iat": now}, strong), now=now)))
    check("milliseconds exp flagged", "exp_milliseconds" in codes(inspect(make_token({"alg": "HS256"}, dict(good_claims, exp=(now + 600) * 1000), strong), now=now)))
    check("long lifetime flagged", "lifetime_long" in codes(inspect(make_token({"alg": "HS256"}, dict(good_claims, exp=now + 30 * 86400), strong), now=now)))
    check("iss mismatch flagged", "iss_mismatch" in codes(inspect(good, now=now, expect_iss="https://evil.example")))
    check("aud list accepted", "aud_mismatch" not in codes(inspect(make_token({"alg": "HS256"}, dict(good_claims, aud=["a", "api://orders"]), strong), now=now, expect_aud="api://orders")))
    check("aud mismatch flagged", "aud_mismatch" in codes(inspect(good, now=now, expect_aud="api://billing")))
    check("missing aud with expected aud is high", "aud_mismatch" in codes(inspect(make_token({"alg": "HS256"}, {k: v for k, v in good_claims.items() if k != "aud"}, strong), now=now, expect_aud="api://orders")))
    check("jku header flagged", "header_jku" in codes(inspect(make_token({"alg": "RS256", "kid": "1", "jku": "https://x/keys"}, good_claims, signature=b"sig"), now=now)))
    check("rs256 without kid is low", "kid_missing" in codes(inspect(make_token({"alg": "RS256"}, good_claims, signature=b"sig"), now=now)))
    check("sensitive claim flagged", "sensitive_claim" in codes(inspect(make_token({"alg": "HS256"}, dict(good_claims, password="hunter2"), strong), now=now)))
    check("empty signature flagged", "signature_empty" in codes(inspect(make_token({"alg": "HS256"}, good_claims), now=now)))
    for bad, label in (("abc.def", "two segments rejected"), ("a.b.c.d.e", "JWE rejected"), ("!!.e30.", "bad base64 rejected")):
        try:
            inspect(bad, now=now)
            check(label, False)
        except TokenError:
            check(label, True)
    check("markdown never prints signature", make_token({"alg": "HS256"}, good_claims, strong).split(".")[2] not in to_markdown(r))
    passed = sum(ok for _, ok in cases)
    for name, ok in cases:
        print(("PASS " if ok else "FAIL ") + name)
    print(f"{passed}/{len(cases)} passed")
    return 0 if passed == len(cases) else 1


def main(argv=None):
    ap = argparse.ArgumentParser(description="Decode and lint a JWT without trusting it.")
    ap.add_argument("token", nargs="?", help="JWT string, or '-' to read from stdin")
    ap.add_argument("--now", help="reference time: ISO 8601 (Z or offset) or epoch seconds; default: current UTC time")
    ap.add_argument("--leeway", type=int, default=0, help="allowed clock skew in seconds (default 0)")
    ap.add_argument("--expect-iss")
    ap.add_argument("--expect-aud")
    ap.add_argument("--max-lifetime", type=int, default=86400, help="max exp-iat seconds before warning (0 = off)")
    ap.add_argument("--secret-env", help="name of an environment variable holding the HS* secret (avoids shell history)")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--strict", action="store_true", help="exit 1 on any finding")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test:
        return self_test()
    if not a.token:
        ap.error("token required (or --self-test)")
    token = sys.stdin.read() if a.token == "-" else a.token
    secret = None
    if a.secret_env:
        secret = os.environ.get(a.secret_env)
        if secret is None:
            print(f"error: environment variable {a.secret_env} is not set", file=sys.stderr)
            return 2
    try:
        r = inspect(token, now=a.now, leeway=a.leeway, expect_iss=a.expect_iss, expect_aud=a.expect_aud,
                    max_lifetime=a.max_lifetime, secret=secret)
    except (TokenError, ValueError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(r, indent=2, ensure_ascii=False) if a.json else to_markdown(r))
    if any(f["severity"] == "high" for f in r["findings"]) or (a.strict and r["findings"]):
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

# Error handling taste of a TypeScript backend engineer
- Fail fast at the edges: validate every external input with a schema library such as Zod before it reaches business logic. Confidence: 0.90
- Separate expected domain errors (not found, conflict, validation) from unexpected bugs, and handle them differently. Confidence: 0.90
- Model expected failures as return values or a Result type; reserve throw for truly exceptional cases. Confidence: 0.70
- Define a small set of typed error classes with a stable machine-readable code field. Confidence: 0.85
- Never swallow an error silently; an empty catch block needs a comment explaining why. Confidence: 0.95
- When rethrowing, wrap the original with the cause option so the stack chain survives. Confidence: 0.85
- Only catch an error where you can add context, recover, or translate it; let everything else bubble up. Confidence: 0.85
- Map errors to HTTP responses in one central error handler, not inside each route. Confidence: 0.90
- Return RFC 9457 problem+json bodies with a code, a human message, and a request id. Confidence: 0.75
- Never leak stack traces, SQL, or internal hostnames to API clients. Confidence: 0.95
- Log each error once, at the boundary that handles it, not at every layer it passes through. Confidence: 0.85
- Include structured context in error logs (request id, user id, operation), never secrets or tokens. Confidence: 0.90
- Retry only idempotent operations, with exponential backoff, jitter, and a hard attempt limit. Confidence: 0.90
- Put a timeout on every network call; a missing timeout is a bug. Confidence: 0.95
- Use AbortController to cancel downstream work when the caller goes away. Confidence: 0.75
- Wrap flaky dependencies in a circuit breaker so one slow service cannot take the whole app down. Confidence: 0.70
- Treat unhandled promise rejections as fatal: log them and let the process restart. Confidence: 0.85
- Let a supervisor restart a crashed process instead of trying to keep a corrupted one alive. Confidence: 0.80
- Report unexpected errors to an error tracker with release and environment tags. Confidence: 0.85
- Write tests for failure paths, not only the happy path. Confidence: 0.85
- Write error messages that say what failed, with which input, and what the caller can do next. Confidence: 0.85
- Use the never type in exhaustive switch statements so new error codes break the build until handled. Confidence: 0.75

# Web security taste of a full-stack developer
- Use the framework's auth or a vetted library; never hand-roll password storage or session handling. Confidence: 0.95
- Hash passwords with Argon2id or bcrypt and a sensible work factor. Confidence: 0.95
- Store sessions in HttpOnly, Secure, SameSite=Lax cookies rather than localStorage. Confidence: 0.90
- Rotate the session id on login and on privilege change. Confidence: 0.85
- Offer passkeys or TOTP as a second factor, and require it for admin accounts. Confidence: 0.85
- Check authorization on the server for every request, including on object ids, to stop IDOR bugs. Confidence: 0.95
- Deny by default: new routes are private until someone explicitly makes them public. Confidence: 0.85
- Use parameterized queries or an ORM everywhere; never build SQL with string concatenation. Confidence: 0.95
- Rely on the template engine's automatic escaping and treat every raw HTML escape hatch as a review flag. Confidence: 0.90
- Ship a strict Content Security Policy with nonces, starting in report-only mode. Confidence: 0.80
- Send HSTS, X-Content-Type-Options: nosniff, and frame-ancestors on every response. Confidence: 0.85
- Protect state-changing requests with SameSite cookies plus CSRF tokens or an origin check. Confidence: 0.85
- Rate-limit login, signup, password reset, and other expensive endpoints. Confidence: 0.90
- Return the same message for an unknown email and a wrong password to avoid account enumeration. Confidence: 0.80
- Validate uploads by content type and size, store them outside the web root, and serve them from a separate domain. Confidence: 0.80
- Block server-side requests to internal and metadata IP ranges to prevent SSRF. Confidence: 0.80
- Keep secrets in a secret manager or environment, never in the repo or the client bundle. Confidence: 0.95
- Run dependency audits and automated update PRs (Dependabot or Renovate) on every repository. Confidence: 0.85
- Log security events such as logins, failed logins, and permission changes, without logging credentials. Confidence: 0.85
- Configure CORS with an explicit origin allowlist; never reflect the request origin with credentials. Confidence: 0.90
- Use short-lived, narrowly scoped API tokens and make them revocable. Confidence: 0.85
- Run a quick OWASP Top 10 check in review for every feature that touches auth, input, or files. Confidence: 0.75

---
name: retry-backoff-policy-calculator
description: Checks retry, backoff and timeout settings across a call chain to prevent retry storms. Computes exponential delay schedules with full, equal or decorrelated jitter, worst-case latency per layer, expected tries and request amplification. Flags inner timeouts that outlive the caller, retries of non-idempotent calls, retries on 4xx, ignored Retry-After and missing caps or deadlines. Includes a stdlib Python calculator with --self-test.
---

# Retry and Backoff Policy Calculator

You help engineers set retry, backoff and timeout policies that make services more reliable instead of turning a small outage into a big one. You collect the settings of every hop in a call chain (browser, gateway, services, databases, third-party APIs), compute what they really do under failure, and propose a safer policy with numbers.

## Files in this skill

- `scripts/retry_policy.py` - reads a call-chain JSON and prints delay schedules, worst-case latency, expected tries, amplification and flags (Python 3 standard library only)
- `references/backoff-math.md` - formulas for backoff, jitter, worst-case time, expected tries, amplification and timeout nesting
- `references/retry-safety-guide.md` - what is safe to retry, idempotency keys, circuit breakers, retry budgets, common library settings
- `templates/retry-policy-review.md` - review report with current chain, findings and proposed policy
- `examples/checkout-chain.json` - a three-layer checkout chain with typical mistakes

## Workflow

### 1. Map the chain
Ask for each hop, outermost first: attempts (total tries), base delay, multiplier, max delay, jitter type, per-try timeout, overall deadline, what it retries on (status codes, "timeout", "network"), whether the call is idempotent, whether an idempotency key is sent, and whether Retry-After is honored. Read them from code or config (SDK clients, HTTP client retry adapters, Envoy/Istio routes, gRPC service config, job queues). Unknown values: use the library default and mark them as assumed.

### 2. Run the calculator
```
python3 scripts/retry_policy.py examples/checkout-chain.json
python3 scripts/retry_policy.py chain.json --failure-rate 0.5 --max-amplification 8
python3 scripts/retry_policy.py chain.json --json > retry-report.json
python3 scripts/retry_policy.py --self-test
```
Exit code 1 means high-severity findings, 2 means bad input.

### 3. Explain the findings
Order: RETRY_UNSAFE and AMPLIFICATION first (they cause duplicate side effects and outages), then INNER_OUTLIVES_CALLER, HOT_LOOP, NO_PER_TRY_TIMEOUT, then medium and low. Explain each with the numbers from the report, for example "one checkout can become 36 calls to the payments provider".

### 4. Propose a fixed policy
Use `references/backoff-math.md` and `references/retry-safety-guide.md`: retry at one layer, full jitter with a cap, inner worst case below the caller's per-try timeout, deadlines passed down, idempotency keys for writes, no retries on 4xx except 408/429, Retry-After honored, a retry budget or circuit breaker. Re-run the script on the proposed JSON and show before and after.

### 5. Write it up
Fill `templates/retry-policy-review.md` and give config snippets for the user's library.

## Example result
For `examples/checkout-chain.json` at 30% failure rate: worst-case amplification 36 calls per checkout (expected 2.74); the gateway layer's 9.3 s worst case outlives the web app's 8 s per-try timeout; the payments layer retries a non-idempotent charge 4 times without an idempotency key and retries 409; no jitter at the gateway; Retry-After ignored on 429/503.

## Rules

- Never recommend retrying a non-idempotent call without an idempotency key.
- Keep numbers honest: label library defaults and guesses as assumptions.
- Prefer fewer retries with good timeouts over many retries.
- Recommend load tests or fault injection to confirm the policy, not just the math.
FILE:references/backoff-math.md
# Backoff and Timeout Math

## Terms
- **attempts**: total tries including the first. attempts 3 = 1 try + 2 retries.
- **base**: first retry delay. **multiplier**: growth factor (usually 2). **cap**: max single delay.
- **per-try timeout**: how long one try may take before it is abandoned.
- **deadline**: total time budget for the call including all tries and delays.

## Delay before retry k (k = 1 .. attempts-1)
d(k) = min(cap, base x multiplier^(k-1))

| Jitter | Delay | Range | Mean |
|---|---|---|---|
| none | d | d | d |
| full | random(0, d) | 0 .. d | d/2 |
| equal | d/2 + random(0, d/2) | d/2 .. d | 0.75 d |
| decorrelated | min(cap, random(base, 3 x previous)) | base .. min(cap, base x 3^k) | simulated |

Full jitter spreads load best and is the default recommendation. Equal jitter keeps a minimum wait. Decorrelated jitter adapts to the previous delay and works well for long-running clients.

## Worst-case duration of one call
worst = attempts x per_try_timeout + sum of max delays, then cut at the deadline if one is set.
Example: 3 tries x 1 s + 100 ms + 200 ms = 3.3 s.

## Expected tries
With an independent failure probability p per try and n attempts:
E[tries] = 1 + p + p^2 + ... + p^(n-1)
p = 0.3, n = 3 -> 1.39. During a full outage p = 1 and E = n.

## Amplification across layers
If layer A retries 3 times and calls B, which retries 3 times and calls C, which retries 4 times, one user request can become 3 x 3 x 4 = 36 calls to C during an outage. This is a retry storm: the retries keep the failing service overloaded, so it cannot recover.

Rules of thumb:
- Retry at one layer only, usually the one closest to the user or the one that owns the idempotency key.
- Inner layers: attempts 1 or 2 with a short deadline.
- Keep worst-case amplification at or below about 10.
- Add a **retry budget**: retries may be at most 10-20% of requests over a sliding window; past that, fail fast.

## Timeout nesting
Each layer's worst case must fit inside its caller's per-try timeout, minus a margin for network and work:
inner_worst <= outer_per_try_timeout - margin
Otherwise the caller abandons the try and retries while the inner layer is still working, which doubles load. Pass a deadline down the chain (for example a `grpc-timeout` or `X-Request-Deadline` header) and stop when it is used up.

## Retry-After
For 429 and 503, servers may send `Retry-After: <seconds>` or an HTTP date. Wait at least that long (use max(Retry-After, computed backoff)), and give up if it is longer than the remaining deadline.
FILE:references/retry-safety-guide.md
# Retry Safety Guide

## What to retry
| Outcome | Retry? | Notes |
|---|---|---|
| Connection refused / reset before the request was sent | Yes | Safe for any method |
| Timeout after the request was sent | Only if idempotent or with an idempotency key | The server may have done the work |
| 408 Request Timeout | Yes, with backoff | |
| 429 Too Many Requests | Yes, after Retry-After | Never faster than the server asks |
| 500 Internal Server Error | Sometimes | Only if the error is known to be transient |
| 502 / 503 / 504 | Yes, with backoff and jitter | Honor Retry-After on 503 |
| 400 / 401 / 403 / 404 / 405 / 410 / 422 | No | Same request fails the same way; fix the request, refresh auth once at most |
| 409 Conflict | Usually no | Exception: an idempotency key that is still in progress, retry after a short wait |

## Idempotency
- GET, HEAD, OPTIONS, PUT and DELETE are idempotent by HTTP semantics, if the server implements them correctly.
- POST and PATCH are not. To retry them, send an **idempotency key** (a UUID per logical operation, e.g. `Idempotency-Key` header) that the server stores with the result for 24 hours or more. Generate the key once, before the first try, and reuse it for every retry.
- Payments, emails, SMS, order creation and stock reservation must never be retried without a key.

## Client checklist
1. Per-try timeout set and shorter than the caller's.
2. Overall deadline set and passed down the chain.
3. Exponential backoff with a cap and full jitter.
4. attempts <= 3 at the edge, 1-2 inside.
5. Retry only on the table's "Yes" rows.
6. Idempotency key on every retried non-idempotent call.
7. Retry budget or circuit breaker so retries stop during an outage.
8. Metrics: tries per request, retry rate, give-up count, by dependency.
9. Log the final error with the attempt count, not every attempt at error level.

## Circuit breaker basics
- Closed: calls flow; count failures in a rolling window.
- Open: after for example 50% failures over at least 20 calls, fail fast for 30 s.
- Half-open: let a few trial calls through; close if they succeed.
A breaker and a retry budget together stop retry storms from deepening an outage.

## Common library settings
- AWS SDKs: standard mode = 3 attempts, full jitter, base 1 s for throttling; adaptive mode adds client-side rate limiting.
- gRPC retry policy: maxAttempts, initialBackoff, maxBackoff, backoffMultiplier, retryableStatusCodes (UNAVAILABLE is the usual one); retries need hedging/transparent retry rules for non-idempotent methods.
- Envoy / Istio: retry_on: 5xx,reset,connect-failure; num_retries; per_try_timeout; set route timeout >= per_try_timeout x (num_retries + 1).
- Python `urllib3.Retry(total=3, backoff_factor=0.5, status_forcelist=[502,503,504], respect_retry_after_header=True, allowed_methods=[...])`.
Check the library docs for the version you use; defaults change.
FILE:templates/retry-policy-review.md
# Retry Policy Review - <service / call chain> - <date>

**Reviewer:** <name>  **Input:** <chain JSON / config files>  **Failure rate assumed:** <30%>

## Chain
| # | Caller -> callee | Attempts | Backoff (base / x / cap) | Jitter | Per-try timeout | Deadline | Worst case |
|---|---|---|---|---|---|---|---|
| 1 | | | | | | | |

**Worst-case amplification:** <36> calls per request (target <= 10)
**Expected amplification at <p>:** <2.7>

## Findings
| Severity | Layer | Code | Problem | Fix |
|---|---|---|---|---|
| high | | | | |

## Proposed policy
| # | Caller -> callee | Attempts | Backoff | Jitter | Per-try timeout | Deadline | Retry on |
|---|---|---|---|---|---|---|---|
| 1 | | | | full | | | |

**New worst-case amplification:** <n>  **New worst-case latency:** <s>

## Follow-ups
- [ ] Idempotency keys for: <calls>
- [ ] Retry budget / circuit breaker for: <dependency>
- [ ] Dashboards: tries per request, retry rate, give-ups
FILE:examples/checkout-chain.json
{
  "name": "checkout: browser -> api gateway -> orders service -> payments provider",
  "failure_rate": 0.3,
  "max_amplification": 10,
  "layers": [
    {
      "name": "web app -> api gateway",
      "attempts": 3,
      "base_delay_ms": 200,
      "multiplier": 2,
      "max_delay_ms": 2000,
      "jitter": "full",
      "per_try_timeout_ms": 8000,
      "deadline_ms": 20000,
      "retry_on": ["network", 502, 503, 504],
      "idempotent": true,
      "respect_retry_after": true
    },
    {
      "name": "api gateway -> orders service",
      "attempts": 3,
      "base_delay_ms": 100,
      "multiplier": 2,
      "max_delay_ms": 1000,
      "jitter": "none",
      "per_try_timeout_ms": 3000,
      "retry_on": [500, 502, 503, 504, "timeout"],
      "idempotent": true
    },
    {
      "name": "orders service -> payments provider (create charge)",
      "attempts": 4,
      "base_delay_ms": 250,
      "multiplier": 2,
      "jitter": "equal",
      "per_try_timeout_ms": 2500,
      "deadline_ms": 9000,
      "retry_on": [409, 429, 500, 503, "timeout"],
      "idempotent": false,
      "idempotency_key": false,
      "respect_retry_after": false
    }
  ]
}
FILE:scripts/retry_policy.py
#!/usr/bin/env python3
"""Retry and backoff policy calculator (Python 3 standard library only).

Input: a JSON file describing a call chain, outermost caller first:
  {"name": "checkout", "failure_rate": 0.3, "max_amplification": 10,
   "layers": [
     {"name": "web -> api", "attempts": 3, "base_delay_ms": 100, "multiplier": 2,
      "max_delay_ms": 2000, "jitter": "full", "per_try_timeout_ms": 5000,
      "deadline_ms": 12000, "retry_on": [502, 503, 504, "timeout"],
      "idempotent": true, "respect_retry_after": true}, ...]}
attempts = total tries including the first one (attempts 1 = no retries).
jitter: none | full | equal | decorrelated.  Delay before retry k (k = 1..attempts-1):
  none  d = min(cap, base * multiplier^(k-1))
  full  U(0, d)              equal  d/2 + U(0, d/2)
  decorrelated  min(cap, U(base, 3 * previous))  (AWS style; expected value simulated, seed 7)

Reports per layer: delay schedule (min / expected / max), worst-case time for one call
(attempts x per-try timeout + max delays, cut at deadline), expected tries at the given
failure rate, and for the chain: worst-case and expected request amplification at the
innermost dependency. Flags:
  NO_JITTER, NO_CAP, HOT_LOOP (base 0), TOO_MANY_ATTEMPTS (> 5), NO_PER_TRY_TIMEOUT,
  NO_DEADLINE, DEADLINE_TOO_SHORT (deadline < one per-try timeout), INNER_OUTLIVES_CALLER
  (worst case > the outer layer's per-try timeout), RETRY_UNSAFE (non-idempotent with
  retries and no idempotency key), RETRY_ON_4XX (retrying 400/401/403/404/409/422),
  IGNORES_RETRY_AFTER (retries 429/503 without honoring Retry-After), AMPLIFICATION.
Usage:
  python3 retry_policy.py chain.json
  python3 retry_policy.py chain.json --failure-rate 0.5 --json
  python3 retry_policy.py --self-test
Exit code: 0 OK, 1 high-severity flags found, 2 bad input.
"""
import argparse
import json
import random
import sys

JITTERS = ("none", "full", "equal", "decorrelated")
BAD_4XX = {400, 401, 403, 404, 405, 409, 410, 422}
SEVERITY = {"INNER_OUTLIVES_CALLER": "high", "RETRY_UNSAFE": "high", "AMPLIFICATION": "high",
            "HOT_LOOP": "high", "NO_PER_TRY_TIMEOUT": "high", "RETRY_ON_4XX": "medium",
            "NO_JITTER": "medium", "NO_CAP": "medium", "IGNORES_RETRY_AFTER": "medium",
            "NO_DEADLINE": "medium", "DEADLINE_TOO_SHORT": "medium", "TOO_MANY_ATTEMPTS": "low"}


class PolicyError(ValueError):
    pass


def num(layer, key, default=None, minimum=0):
    v = layer.get(key, default)
    if v is None:
        return None
    if isinstance(v, bool) or not isinstance(v, (int, float)):
        raise PolicyError(f"layer {layer.get('name', '?')!r}: {key} must be a number")
    if v < minimum:
        raise PolicyError(f"layer {layer.get('name', '?')!r}: {key} must be >= {minimum}")
    return v


def delays(layer):
    """Return [(min, expected, max)] in ms for each retry."""
    n = int(layer["attempts"])
    base, mult, cap = layer["base_delay_ms"], layer["multiplier"], layer["max_delay_ms"]
    capf = (lambda x: x) if cap is None else (lambda x: min(cap, x))
    j = layer["jitter"]
    out = []
    if j == "decorrelated":
        rng = random.Random(7)
        runs = 20000
        sums = [0.0] * (n - 1)
        for _ in range(runs):
            prev = base
            for k in range(n - 1):
                prev = capf(rng.uniform(base, prev * 3)) if base > 0 else 0.0
                sums[k] += prev
        hi = base
        for k in range(n - 1):
            hi = capf(hi * 3)
            out.append((capf(base), sums[k] / runs, hi))
        return out
    for k in range(n - 1):
        d = capf(base * mult ** k)
        if j == "none":
            out.append((d, d, d))
        elif j == "full":
            out.append((0.0, d / 2, d))
        else:
            out.append((d / 2, d * 0.75, d))
    return out


def normalize(raw):
    layers = raw.get("layers") if isinstance(raw, dict) else None
    if not layers or not isinstance(layers, list):
        raise PolicyError("JSON must have a non-empty 'layers' list (outermost caller first)")
    out = []
    for i, l in enumerate(layers):
        if not isinstance(l, dict):
            raise PolicyError(f"layer {i} must be an object")
        l = dict(l)
        l.setdefault("name", f"layer {i + 1}")
        a = num(l, "attempts", 1, 1)
        if int(a) != a:
            raise PolicyError(f"layer {l['name']!r}: attempts must be a whole number")
        l["attempts"] = int(a)
        l["base_delay_ms"] = num(l, "base_delay_ms", 0)
        l["multiplier"] = num(l, "multiplier", 2, 1)
        l["max_delay_ms"] = num(l, "max_delay_ms", None)
        l["per_try_timeout_ms"] = num(l, "per_try_timeout_ms", None, 1)
        l["deadline_ms"] = num(l, "deadline_ms", None, 1)
        j = str(l.get("jitter", "none")).lower()
        if j not in JITTERS:
            raise PolicyError(f"layer {l['name']!r}: jitter must be one of {', '.join(JITTERS)}")
        l["jitter"] = j
        l["retry_on"] = list(l.get("retry_on", []))
        l["idempotent"] = bool(l.get("idempotent", True))
        l["idempotency_key"] = bool(l.get("idempotency_key", False))
        l["respect_retry_after"] = bool(l.get("respect_retry_after", False))
        out.append(l)
    return out


def expected_tries(n, p):
    if p >= 1:
        return float(n)
    return sum(p ** k for k in range(n))


def analyze(raw, failure_rate=None, max_amp=None):
    layers = normalize(raw)
    p = failure_rate if failure_rate is not None else raw.get("failure_rate", 0.2)
    if not 0 <= p <= 1:
        raise PolicyError("failure rate must be between 0 and 1")
    max_amp = max_amp if max_amp is not None else raw.get("max_amplification", 10)
    res = {"name": raw.get("name", "retry policy"), "failure_rate": p, "max_amplification": max_amp, "layers": []}
    worst_amp, exp_amp = 1, 1.0
    for i, l in enumerate(layers):
        n = l["attempts"]
        ds = delays(l)
        flags = []
        retries = n > 1
        if retries and l["jitter"] == "none":
            flags.append(("NO_JITTER", "all clients retry at the same moments; use full or decorrelated jitter"))
        if retries and l["max_delay_ms"] is None and n > 3:
            flags.append(("NO_CAP", "no max_delay_ms; exponential delays grow without bound"))
        if retries and l["base_delay_ms"] == 0:
            flags.append(("HOT_LOOP", "base_delay_ms is 0; retries fire immediately and hammer the dependency"))
        if n > 5:
            flags.append(("TOO_MANY_ATTEMPTS", f"{n} attempts; 3 (2 retries) is usually enough"))
        if l["per_try_timeout_ms"] is None:
            flags.append(("NO_PER_TRY_TIMEOUT", "no per-try timeout; one hung call can block every retry"))
        if l["deadline_ms"] is None and retries:
            flags.append(("NO_DEADLINE", "no overall deadline across attempts"))
        if l["deadline_ms"] is not None and l["per_try_timeout_ms"] and l["deadline_ms"] < l["per_try_timeout_ms"]:
            flags.append(("DEADLINE_TOO_SHORT", "deadline is shorter than a single try"))
        if retries and not l["idempotent"] and not l["idempotency_key"]:
            flags.append(("RETRY_UNSAFE", "retries a non-idempotent call without an idempotency key (double charge / duplicate write risk)"))
        bad = sorted(c for c in l["retry_on"] if isinstance(c, int) and c in BAD_4XX)
        if retries and bad:
            flags.append(("RETRY_ON_4XX", f"retries client errors {bad}; they will fail the same way again"))
        if retries and any(c in (429, 503) for c in l["retry_on"]) and not l["respect_retry_after"]:
            flags.append(("IGNORES_RETRY_AFTER", "retries 429/503 but ignores the Retry-After header"))
        max_delay_total = sum(d[2] for d in ds)
        exp_delay_total = sum(d[1] for d in ds)
        worst = None
        if l["per_try_timeout_ms"] is not None:
            worst = n * l["per_try_timeout_ms"] + max_delay_total
            if l["deadline_ms"] is not None:
                worst = min(worst, l["deadline_ms"])
        elif l["deadline_ms"] is not None:
            worst = l["deadline_ms"]
        if i > 0:
            parent = layers[i - 1]
            pt = parent["per_try_timeout_ms"]
            if pt is not None and (worst is None or worst > pt):
                flags.append(("INNER_OUTLIVES_CALLER", f"worst case {fmt_ms(worst)} > caller per-try timeout {fmt_ms(pt)}; the caller gives up and retries while this layer is still working"))
        et = expected_tries(n, p)
        worst_amp *= n
        exp_amp *= et
        res["layers"].append({
            "name": l["name"], "attempts": n, "jitter": l["jitter"],
            "delays_ms": [[round(a, 1), round(b, 1), round(c, 1)] for a, b, c in ds],
            "max_delay_total_ms": round(max_delay_total, 1), "expected_delay_total_ms": round(exp_delay_total, 1),
            "per_try_timeout_ms": l["per_try_timeout_ms"], "deadline_ms": l["deadline_ms"],
            "worst_case_ms": None if worst is None else round(worst, 1),
            "expected_tries": round(et, 3), "cumulative_worst_amplification": worst_amp,
            "cumulative_expected_amplification": round(exp_amp, 3),
            "flags": [{"code": c, "severity": SEVERITY[c], "detail": d} for c, d in flags]})
    if worst_amp > max_amp:
        res["layers"][-1]["flags"].append({"code": "AMPLIFICATION", "severity": "high",
            "detail": f"one top-level request can become {worst_amp} calls to the innermost dependency during an outage (limit {max_amp}); retry at one layer only or add a retry budget"})
    res["worst_amplification"] = worst_amp
    res["expected_amplification"] = round(exp_amp, 3)
    res["high_flags"] = sum(1 for l in res["layers"] for f in l["flags"] if f["severity"] == "high")
    return res


def fmt_ms(v):
    if v is None:
        return "unbounded"
    if v >= 1000:
        s = f"{v / 1000:.2f}".rstrip("0").rstrip(".")
        return s + " s"
    return f"{v:g} ms"


def to_markdown(r):
    out = [f"# Retry policy report: {r['name']}", "",
           f"Failure rate used for expected values: {r['failure_rate']:.0%}", "",
           "| Layer | Attempts | Jitter | Max total delay | Per-try timeout | Deadline | Worst case | Expected tries |",
           "|---|---|---|---|---|---|---|---|"]
    for l in r["layers"]:
        out.append(f"| {l['name']} | {l['attempts']} | {l['jitter']} | {fmt_ms(l['max_delay_total_ms'])} | "
                   f"{fmt_ms(l['per_try_timeout_ms'])} | {fmt_ms(l['deadline_ms'])} | {fmt_ms(l['worst_case_ms'])} | {l['expected_tries']:g} |")
    out += ["", "## Delay schedule (min / expected / max)"]
    for l in r["layers"]:
        if not l["delays_ms"]:
            out.append(f"- {l['name']}: no retries")
            continue
        parts = [f"retry {k + 1}: {fmt_ms(a)} / {fmt_ms(b)} / {fmt_ms(c)}" for k, (a, b, c) in enumerate(l["delays_ms"])]
        out.append(f"- {l['name']}: " + "; ".join(parts))
    out += ["", "## Amplification at the innermost dependency",
            f"- Worst case (every try fails): {r['worst_amplification']} calls per top-level request",
            f"- Expected at {r['failure_rate']:.0%} failure rate: {r['expected_amplification']:g} calls per top-level request", "",
            "## Findings"]
    n = 0
    for sev in ("high", "medium", "low"):
        for l in r["layers"]:
            for f in l["flags"]:
                if f["severity"] == sev:
                    n += 1
                    out.append(f"- **{sev.upper()} {f['code']}** ({l['name']}): {f['detail']}")
    if not n:
        out.append("- No findings.")
    return "\n".join(out)


def self_test():
    cases = []

    def ok(name, cond):
        cases.append((name, bool(cond)))

    def one(**kw):
        base = {"name": "x", "attempts": 3, "base_delay_ms": 100, "multiplier": 2, "max_delay_ms": 1000,
                "jitter": "full", "per_try_timeout_ms": 1000, "deadline_ms": 5000, "retry_on": [503],
                "respect_retry_after": True}
        base.update(kw)
        return base

    def codes(r, i=0):
        return {f["code"] for f in r["layers"][i]["flags"]}

    d = delays(normalize({"layers": [one(jitter="none", attempts=5)]})[0])
    ok("exponential no jitter 100,200,400,800", [x[2] for x in d] == [100, 200, 400, 800])
    d = delays(normalize({"layers": [one(jitter="none", attempts=6)]})[0])
    ok("cap applied at 1000", d[-1][2] == 1000 and d[-2][2] == 800)
    d = delays(normalize({"layers": [one(jitter="full")]})[0])
    ok("full jitter mean half", d[0] == (0.0, 50.0, 100))
    d = delays(normalize({"layers": [one(jitter="equal")]})[0])
    ok("equal jitter 50..100 mean 75", d[0] == (50.0, 75.0, 100))
    d = delays(normalize({"layers": [one(jitter="decorrelated", attempts=4)]})[0])
    ok("decorrelated within bounds", all(a <= b <= c for a, b, c in d) and d[0][2] == 300 and d[2][2] == 1000)
    r = analyze({"layers": [one()]})
    ok("clean policy has no flags", not codes(r))
    ok("worst case = 3x1000 + 100 + 200", r["layers"][0]["worst_case_ms"] == 3300)
    r = analyze({"layers": [one(deadline_ms=2000)]})
    ok("deadline caps worst case", r["layers"][0]["worst_case_ms"] == 2000)
    ok("expected tries p=0.5 n=3 is 1.75", expected_tries(3, 0.5) == 1.75 and expected_tries(3, 1) == 3)
    ok("no jitter flagged", "NO_JITTER" in codes(analyze({"layers": [one(jitter="none")]})))
    ok("hot loop flagged", "HOT_LOOP" in codes(analyze({"layers": [one(base_delay_ms=0)]})))
    ok("no cap flagged with 4+ attempts", "NO_CAP" in codes(analyze({"layers": [one(max_delay_ms=None, attempts=4)]})))
    ok("unsafe retry flagged", "RETRY_UNSAFE" in codes(analyze({"layers": [one(idempotent=False)]})))
    ok("idempotency key makes it safe", "RETRY_UNSAFE" not in codes(analyze({"layers": [one(idempotent=False, idempotency_key=True)]})))
    ok("4xx retry flagged", "RETRY_ON_4XX" in codes(analyze({"layers": [one(retry_on=[400, 503])]})))
    ok("Retry-After ignored flagged", "IGNORES_RETRY_AFTER" in codes(analyze({"layers": [one(retry_on=[429], respect_retry_after=False)]})))
    ok("single attempt is not a retry policy", not codes(analyze({"layers": [one(attempts=1, jitter="none", deadline_ms=None)]})))
    r = analyze({"layers": [one(per_try_timeout_ms=2000, deadline_ms=None), one(per_try_timeout_ms=1000, deadline_ms=None)]})
    ok("inner outlives caller flagged", "INNER_OUTLIVES_CALLER" in codes(r, 1))
    r = analyze({"layers": [one(per_try_timeout_ms=4000), one(per_try_timeout_ms=1000, deadline_ms=3500)]})
    ok("inner within caller timeout ok", "INNER_OUTLIVES_CALLER" not in codes(r, 1))
    r = analyze({"layers": [one(), one(), one()], "max_amplification": 10})
    ok("3x3x3 amplification 27 flagged", r["worst_amplification"] == 27 and "AMPLIFICATION" in codes(r, 2))
    r = analyze({"layers": [one(), one()]}, failure_rate=0.5)
    ok("expected amplification 1.75^2", r["expected_amplification"] == 3.062)
    for bad, label in ((({"layers": []}), "empty layers rejected"), (({"layers": [one(jitter="random")]}), "bad jitter rejected"),
                       (({"layers": [one(attempts=0)]}), "zero attempts rejected"), (({"layers": [one(attempts=2.5)]}), "fractional attempts rejected")):
        try:
            analyze(bad)
            ok(label, False)
        except PolicyError:
            ok(label, True)
    ok("fmt_ms", fmt_ms(1500) == "1.5 s" and fmt_ms(250) == "250 ms" and fmt_ms(None) == "unbounded")
    passed = sum(c for _, c in cases)
    for name, c in cases:
        print(("PASS " if c else "FAIL ") + name)
    print(f"{passed}/{len(cases)} passed")
    return 0 if passed == len(cases) else 1


def main(argv=None):
    ap = argparse.ArgumentParser(description="Check retry/backoff/timeout policies across a call chain.")
    ap.add_argument("file", nargs="?", help="chain JSON ('-' for stdin)")
    ap.add_argument("--failure-rate", type=float, help="probability that one try fails (default from file or 0.2)")
    ap.add_argument("--max-amplification", type=float, help="flag chains above this worst-case amplification (default 10)")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test:
        return self_test()
    if not a.file:
        ap.error("chain JSON required (or --self-test)")
    try:
        raw = json.load(sys.stdin if a.file == "-" else open(a.file, encoding="utf-8"))
        r = analyze(raw, a.failure_rate, a.max_amplification)
    except (OSError, PolicyError, ValueError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(r, indent=2) if a.json else to_markdown(r))
    return 1 if r["high_flags"] else 0


if __name__ == "__main__":
    sys.exit(main())

---
name: http-cache-header-reviewer
description: Reviews HTTP caching headers (Cache-Control, Expires, ETag, Last-Modified, Vary, Set-Cookie) for pages, assets, APIs. Computes browser and CDN freshness, flags cookie or personal responses a CDN may share, HTML cached too long, hashed assets cached too briefly, immutable on changing URLs, Vary fragmentation, missing validators, heuristic caching and cached errors. Suggests a header per resource. Includes a stdlib Python checker with --self-test.
---

# HTTP Cache Header Reviewer

You help web developers get caching right: fast repeat visits and low CDN bills without stale pages after a deploy or, worse, one user's data served to another. You collect real response headers, explain what each cache will actually do with them, and propose a header per resource type with a short fix list.

## Files in this skill

- `scripts/cache_audit.py` - reads response blocks (curl -sI or DevTools output) and prints resource kind, browser and CDN TTL, revalidation, flags and suggested Cache-Control (Python 3 standard library only)
- `references/cache-control-directives.md` - every directive, freshness and heuristic rules, validators, Vary, cacheable status codes
- `references/caching-strategy-playbook.md` - recommended headers per resource, deploy safety, CDN notes, how to collect and verify headers
- `templates/cache-review-report.md` - report with current vs recommended headers and a fix list
- `examples/shop-responses.txt` - eight responses from a sample shop with common mistakes

## Workflow

### 1. Collect headers
Ask for a list of representative URLs: the home page, a logged-in page, a hashed JS/CSS file, an unhashed script, an image, a font, a public API call, a personal API call, an error response. Capture with `curl -sI <url>` (add `-H 'Cookie: ...'` or `-H 'Authorization: ...'` for logged-in requests, never paste real tokens into the report) or DevTools. Put each in the block format of `examples/shop-responses.txt`: a `### METHOD URL` line, optional `> Request-Header: value` lines, the status line, then the response headers.

### 2. Run the checker
```
python3 scripts/cache_audit.py examples/shop-responses.txt
python3 scripts/cache_audit.py headers.txt --json > cache-report.json
python3 scripts/cache_audit.py --self-test
```
Exit code 1 means high-severity findings, 2 means the input could not be parsed.

### 3. Explain the findings
Order: privacy first (AUTH_SHARED_CACHE, PUBLIC_WITH_COOKIE, PERSONAL_NOT_PRIVATE), then correctness after deploys (HTML_LONG_TTL, IMMUTABLE_UNHASHED, ERROR_CACHED), then performance (HASHED_SHORT_TTL, ASSET_NO_CACHING, VARY_FRAGMENTS, NO_VALIDATOR, HEURISTIC_ONLY) and hygiene (INVALID_DIRECTIVE, NO_STORE_CONFLICT, EXPIRES_ONLY, PRAGMA_ONLY). Use `references/cache-control-directives.md` to explain why.

### 4. Recommend headers
Use `references/caching-strategy-playbook.md`. Give the exact header per resource and where to set it in the user's stack (Nginx `add_header`, Express `res.set`, Next.js `headers()`, S3/CloudFront metadata, Cloudflare rules). Mention the deploy rule: hashed assets for a year, HTML revalidates.

### 5. Verify and report
Fill `templates/cache-review-report.md`. After changes, collect headers again and re-run the checker; check CDN HIT/MISS headers.

## Example result
For `examples/shop-responses.txt`: 5 high findings (home page HTML cached 1 day, widget.js immutable without a hash, the cart API with a session cookie cached by the CDN for 2 minutes, the login page with Set-Cookie marked public and cached 10 minutes), 4 medium (app.3f9a1c7b.js cached only 1 hour, hero.jpg on an 8.1-day heuristic, Vary: Cookie, a 503 cached for 1 hour) and 1 low (cart API has no validator); only vendor.81d2e0aa.css is already ideal.

## Rules

- Treat any chance of a shared cache storing personal data as the top issue.
- Do not ask for or keep real cookies or tokens; placeholders are enough.
- Recommend purges or URL versioning when changing TTLs that are already live.
- Say when behavior depends on the CDN's configuration and suggest checking it.
FILE:references/cache-control-directives.md
# Cache-Control Directives and Freshness Rules

## Response directives
| Directive | Meaning |
|---|---|
| max-age=N | Fresh for N seconds in any cache (browser and shared) |
| s-maxage=N | Overrides max-age for shared caches (CDN, proxy) only |
| no-cache | May be stored, but must be revalidated with the origin before every use |
| no-store | Must not be stored anywhere; overrides everything else |
| private | Only the browser may store it; CDNs and proxies must not |
| public | Shared caches may store it even if it would normally not be cacheable (for example with Authorization) |
| must-revalidate | Once stale, must not be served without a successful revalidation |
| proxy-revalidate | Same as must-revalidate, for shared caches only |
| immutable | Will never change while fresh; browsers skip revalidation on reload |
| stale-while-revalidate=N | May serve stale for N seconds while refreshing in the background |
| stale-if-error=N | May serve stale for N seconds if the origin returns 5xx or is unreachable |
| no-transform | Intermediaries must not modify the body (recompress images, etc.) |

Common mistakes: `no-cache` does NOT mean "do not cache" (that is `no-store`); `max-age` values must be whole seconds (`max-age=10m` is invalid and ignored); `Pragma: no-cache` only works on requests.

## Freshness lifetime (RFC 9111)
1. Shared cache: s-maxage, else max-age, else Expires - Date.
2. Browser: max-age, else Expires - Date.
3. Neither present and Last-Modified present: caches may use a heuristic, usually 10% of (Date - Last-Modified). A file last changed 80 days ago may be reused for 8 days with no check.
4. `private` sets the shared lifetime to 0. `no-store` sets both to 0.

## Validators and revalidation
- ETag (strong `"abc"` or weak `W/"abc"`) -> client sends If-None-Match.
- Last-Modified -> client sends If-Modified-Since.
- A match returns `304 Not Modified` with no body, which is cheap. Without a validator, every revalidation downloads the whole body.

## Vary
- `Vary: Accept-Encoding` is normal and safe.
- `Vary: Cookie`, `User-Agent` or `Referer` creates one cache entry per distinct value, which usually means no CDN hits.
- `Vary: *` means the response can never be reused.

## Status codes
Cacheable by default when fresh: 200, 203, 204, 206, 300, 301, 308, 404, 405, 410, 414, 501. Error responses (4xx/5xx) should have short or no caching, or a CDN may serve an outage page for hours. Use `stale-if-error` to serve the last good copy instead.
FILE:references/caching-strategy-playbook.md
# Caching Strategy Playbook

## Recommended headers by resource
| Resource | Cache-Control | Notes |
|---|---|---|
| HTML pages, app shell | `no-cache` + ETag | Always fresh after deploys; 304s keep it cheap |
| Marketing HTML on a CDN | `public, max-age=0, s-maxage=300, stale-while-revalidate=60` | Purge the CDN on publish |
| JS/CSS with a content hash (`app.3f9a1c7b.js`) | `public, max-age=31536000, immutable` | New build = new URL |
| JS/CSS without a hash | `public, max-age=300, stale-while-revalidate=86400` | Or add hashing to the build |
| Images, user uploads with stable URLs | `public, max-age=86400, stale-while-revalidate=604800` | Version the URL when the image changes |
| Fonts (versioned URL) | `public, max-age=31536000, immutable` | Also send CORS headers |
| Public API data (prices, catalog) | `public, max-age=30, s-maxage=300, stale-if-error=600` | Keep TTLs short; purge on change |
| Personal API data (cart, profile) | `private, no-cache` + ETag | Never public |
| Secrets, banking, health records, one-time tokens | `no-store` | Also on the logout and account pages |
| Redirect 301 | `max-age=86400` at first | Browsers can cache a bare 301 forever |
| Errors 5xx | `no-store` or `max-age=0`, use stale-if-error on the good responses | |

## Deploy safety
1. Hash every built asset name, then cache it for a year.
2. Make HTML revalidate so it always points at the current hashes.
3. Keep old hashed assets online for at least one release so open tabs still work.
4. Never put `immutable` on a URL whose content can change.

## CDN notes
- Most CDNs ignore `private`/`no-store` only if misconfigured; check "respect origin headers".
- Many CDNs do not cache responses with Set-Cookie by default; some do if `public` is set. Strip Set-Cookie from static responses.
- Use a `Surrogate-Control` or CDN-Cache-Control header if the CDN supports it and you want a CDN TTL different from the browser TTL.
- Normalize the cache key (ignore tracking query strings like utm_*).

## How to collect headers
```
curl -sI https://example.com/ ; curl -sI -H 'Cookie: session=...' https://example.com/api/me
curl -s -o /dev/null -D - https://example.com/assets/app.js   # GET instead of HEAD
```
Some servers answer HEAD differently from GET; when in doubt use the GET form. In Chrome DevTools: Network tab, select a request, Headers, Response Headers (copy as text).

## Checking the result
- Reload twice in DevTools: hashed assets should show "(memory cache)" or "(disk cache)", HTML should show 304 or 200.
- CDN: look at its `Age`, `X-Cache` or `CF-Cache-Status` header (HIT / MISS / BYPASS).
FILE:templates/cache-review-report.md
# HTTP Cache Review - <site / service> - <date>

**Scope:** <URLs or HAR file>  **CDN:** <name or none>  **Collected with:** <curl -sI / DevTools / HAR>

## Summary
- Responses reviewed: <n>  High: <n>  Medium: <n>  Low: <n>
- Biggest risk: <for example personal API cached on the CDN>
- Biggest win: <for example year-long caching for hashed assets>

## Current vs recommended
| Resource | Kind | Current Cache-Control | Browser / CDN TTL | Flags | Recommended |
|---|---|---|---|---|---|
| | | | | | |

## Fix list
| Priority | Change | Where (server, framework, CDN rule) | Owner | Done |
|---|---|---|---|---|
| 1 | | | | [ ] |

## Verification
- [ ] Re-run `python3 scripts/cache_audit.py` on fresh headers: no high findings
- [ ] CDN shows HIT for static files, BYPASS/MISS for personal API
- [ ] Deploy test: new HTML loads new assets without a hard refresh
FILE:examples/shop-responses.txt
# Responses captured with: curl -sI <url>  (request headers sent are listed with "> ")
### GET https://shop.example/
HTTP/2 200
content-type: text/html; charset=utf-8
cache-control: public, max-age=86400
date: Wed, 21 Oct 2026 07:00:00 GMT

### GET https://shop.example/assets/app.3f9a1c7b.js
HTTP/2 200
content-type: application/javascript
cache-control: public, max-age=3600
etag: "3f9a1c7b"

### GET https://shop.example/assets/vendor.81d2e0aa.css
HTTP/2 200
content-type: text/css
cache-control: public, max-age=31536000, immutable
etag: "81d2e0aa"

### GET https://shop.example/js/widget.js
HTTP/2 200
content-type: application/javascript
cache-control: public, max-age=31536000, immutable

### GET https://shop.example/img/hero.jpg
HTTP/2 200
content-type: image/jpeg
date: Wed, 21 Oct 2026 07:00:00 GMT
last-modified: Sat, 01 Aug 2026 07:00:00 GMT

### GET https://shop.example/api/cart
> Cookie: session=abc123
HTTP/2 200
content-type: application/json
cache-control: public, s-maxage=120
vary: Cookie

### POST https://shop.example/login
HTTP/2 200
content-type: text/html; charset=utf-8
cache-control: public, max-age=600
set-cookie: session=xyz789; HttpOnly; Secure; SameSite=Lax

### GET https://shop.example/api/products?page=1
HTTP/2 503
content-type: application/json
cache-control: public, max-age=3600
FILE:scripts/cache_audit.py
#!/usr/bin/env python3
"""HTTP caching header reviewer (Python 3 standard library only).

Input: a text file with one block per response, as copied from `curl -sI`, browser
devtools or a HAR export:
  ### GET https://shop.example/assets/app.3f9a1c7b.js
  > Authorization: Bearer ...          (optional request headers start with "> ")
  HTTP/2 200
  content-type: application/javascript
  cache-control: public, max-age=31536000, immutable
  etag: "3f9a1c7b"
Blank lines and lines starting with "#" (but not "###") are ignored.

For each response it works out the kind (html, hashed-asset, asset, api, image, font, other),
browser and shared-cache (CDN) freshness in seconds (s-maxage > max-age > Expires - Date >
10% heuristic from Last-Modified), whether it can be revalidated, and flags:
  NO_STORE_CONFLICT, PUBLIC_WITH_COOKIE, AUTH_SHARED_CACHE, PERSONAL_NOT_PRIVATE, VARY_STAR,
  VARY_FRAGMENTS, HTML_LONG_TTL, HASHED_SHORT_TTL, IMMUTABLE_UNHASHED, ASSET_NO_CACHING,
  NO_VALIDATOR, HEURISTIC_ONLY, EXPIRES_ONLY, PRAGMA_ONLY, ERROR_CACHED, INVALID_DIRECTIVE.
It suggests a Cache-Control value per kind.
Usage:
  python3 cache_audit.py responses.txt
  python3 cache_audit.py responses.txt --json
  python3 cache_audit.py --self-test
Exit code: 0 OK, 1 high-severity flags found, 2 bad input.
"""
import argparse
import email.utils
import json
import re
import sys

KNOWN = {"max-age", "s-maxage", "no-cache", "no-store", "no-transform", "must-revalidate",
         "proxy-revalidate", "must-understand", "private", "public", "immutable",
         "stale-while-revalidate", "stale-if-error"}
NEEDS_INT = {"max-age", "s-maxage", "stale-while-revalidate", "stale-if-error"}
HASH_RE = re.compile(r"[.\-_]([0-9a-f]{8,}|[0-9A-Za-z_-]{8,}(?=\.[a-z0-9]+$))\.[a-z0-9]+$")
SEVERITY = {"NO_STORE_CONFLICT": "low", "PUBLIC_WITH_COOKIE": "high", "AUTH_SHARED_CACHE": "high",
            "PERSONAL_NOT_PRIVATE": "medium", "VARY_STAR": "medium", "VARY_FRAGMENTS": "medium",
            "HTML_LONG_TTL": "high", "HASHED_SHORT_TTL": "medium", "IMMUTABLE_UNHASHED": "high",
            "ASSET_NO_CACHING": "medium", "NO_VALIDATOR": "low", "HEURISTIC_ONLY": "medium",
            "EXPIRES_ONLY": "low", "PRAGMA_ONLY": "low", "ERROR_CACHED": "medium",
            "INVALID_DIRECTIVE": "medium"}
RECOMMEND = {
    "html": "no-cache (plus ETag or Last-Modified), or max-age=0, must-revalidate",
    "hashed-asset": "public, max-age=31536000, immutable",
    "asset": "public, max-age=3600, stale-while-revalidate=86400 (or add a content hash to the file name)",
    "image": "public, max-age=86400, stale-while-revalidate=604800",
    "font": "public, max-age=31536000, immutable (if the URL is versioned)",
    "api": "private, no-cache for user data; public, max-age=60, s-maxage=300 only for truly public data",
    "personal": "private, no-cache (or no-store for sensitive data)",
    "other": "set an explicit Cache-Control",
}


class CacheError(ValueError):
    pass


def parse_cc(value):
    """Return (directives dict, problems list)."""
    d, problems = {}, []
    for part in [p.strip() for p in value.split(",") if p.strip()]:
        k, eq, v = part.partition("=")
        k = k.strip().lower()
        v = v.strip().strip('"')
        if k not in KNOWN:
            problems.append(f"unknown directive {k!r}")
            continue
        if k in NEEDS_INT:
            if not re.fullmatch(r"\d+", v or ""):
                problems.append(f"{k} needs a whole number of seconds, got {v!r}")
                continue
            d[k] = int(v)
        else:
            d[k] = True if not eq else v
    return d, problems


def parse_blocks(text):
    blocks, cur = [], None
    for raw in text.splitlines():
        line = raw.rstrip("\r")
        if line.startswith("###"):
            cur = {"label": line[3:].strip(), "status": None, "headers": {}, "request": {}}
            blocks.append(cur)
            continue
        if not line.strip() or line.lstrip().startswith("#"):
            continue
        if cur is None:
            raise CacheError("file must start with a '### METHOD URL' line")
        if line.startswith(">"):
            k, _, v = line[1:].strip().partition(":")
            cur["request"][k.strip().lower()] = v.strip()
            continue
        m = re.match(r"HTTP/[\d.]+\s+(\d{3})", line)
        if m:
            cur["status"] = int(m.group(1))
            continue
        k, colon, v = line.partition(":")
        if not colon:
            raise CacheError(f"cannot parse header line {line!r} in block {cur['label']!r}")
        k = k.strip().lower()
        if k in cur["headers"] and k in ("cache-control", "vary"):
            cur["headers"][k] += ", " + v.strip()
        elif k == "set-cookie":
            cur["headers"].setdefault(k, []).append(v.strip())
        else:
            cur["headers"][k] = v.strip()
    if not blocks:
        raise CacheError("no response blocks found")
    return blocks


def url_of(label):
    parts = label.split()
    return parts[-1] if parts else ""


def kind_of(url, ctype):
    path = re.sub(r"[?#].*$", "", url)
    name = path.rsplit("/", 1)[-1].lower()
    ctype = ctype.lower()
    hashed = bool(HASH_RE.search(name)) and bool(re.search(r"\d", name))
    if "text/html" in ctype or name.endswith(".html") or (not ctype and "." not in name):
        return "html", False
    if "json" in ctype or "/api/" in path or "graphql" in path:
        return "api", False
    if name.endswith((".woff2", ".woff", ".ttf", ".otf")) or "font" in ctype:
        return "font", hashed
    if ctype.startswith("image/") or name.endswith((".png", ".jpg", ".jpeg", ".gif", ".webp", ".avif", ".svg", ".ico")):
        return ("hashed-asset" if hashed else "image"), hashed
    if name.endswith((".js", ".mjs", ".css", ".map", ".wasm")) or "javascript" in ctype or "text/css" in ctype:
        return ("hashed-asset" if hashed else "asset"), hashed
    return "other", hashed


def http_date(v):
    try:
        return email.utils.parsedate_to_datetime(v)
    except (TypeError, ValueError, IndexError):
        return None


def review(block):
    h, req = block["headers"], block["request"]
    status = block["status"] or 200
    url = url_of(block["label"])
    kind, hashed = kind_of(url, h.get("content-type", ""))
    cc, problems = parse_cc(h.get("cache-control", ""))
    flags = []

    def flag(code, detail):
        flags.append({"code": code, "severity": SEVERITY[code], "detail": detail})

    for p in problems:
        flag("INVALID_DIRECTIVE", p)
    cookies = h.get("set-cookie", [])
    auth = "authorization" in req or "cookie" in req
    date, expires, lastmod = http_date(h.get("date")), h.get("expires"), http_date(h.get("last-modified"))
    no_store = "no-store" in cc
    heuristic = False
    if no_store:
        browser = shared = 0
    else:
        browser = cc.get("max-age")
        if browser is None and expires is not None:
            ex = http_date(expires)
            browser = max(0, int((ex - date).total_seconds())) if ex and date else 0
        if browser is None and "no-cache" not in cc and lastmod and date and status in (200, 203, 204, 206, 300, 301, 404, 405, 410, 414, 501):
            browser = max(0, int((date - lastmod).total_seconds() / 10))
            heuristic = True
        browser = browser or 0
        shared = 0 if "private" in cc else cc.get("s-maxage", browser)
    if "no-cache" in cc:
        browser_eff, shared_eff = 0, 0
    else:
        browser_eff, shared_eff = browser, shared
    validator = "etag" in h or "last-modified" in h
    personal = bool(cookies) or auth

    if no_store and any(k in cc for k in ("max-age", "s-maxage", "public", "immutable")):
        flag("NO_STORE_CONFLICT", "no-store overrides the other directives; remove the extras so intent is clear")
    if cookies and ("public" in cc or "s-maxage" in cc) and not no_store:
        flag("PUBLIC_WITH_COOKIE", "response sets a cookie but allows shared caches; a CDN may serve one user's cookie to others")
    if auth and shared > 0 and ("public" in cc or "s-maxage" in cc) and kind in ("api", "html", "other"):
        flag("AUTH_SHARED_CACHE", "authenticated request with a response that shared caches may store; personal data can leak across users")
    elif personal and kind in ("api", "html", "other") and not no_store and "private" not in cc \
            and not any(f["code"] == "PUBLIC_WITH_COOKIE" for f in flags):
        flag("PERSONAL_NOT_PRIVATE", "personalized response without private or no-store")
    vary = [v.strip().lower() for v in h.get("vary", "").split(",") if v.strip()]
    if "*" in vary:
        flag("VARY_STAR", "Vary: * makes the response effectively uncacheable")
    frag = [v for v in vary if v in ("user-agent", "cookie", "referer")]
    if frag and shared > 0:
        flag("VARY_FRAGMENTS", f"Vary on {', '.join(frag)} splits the CDN cache into many copies; vary on a normalized header instead")
    if kind == "html" and browser_eff > 300:
        flag("HTML_LONG_TTL", f"HTML cached {fmt_s(browser_eff)} without revalidation; users keep old pages that point to deleted assets after a deploy")
    if kind == "hashed-asset" and browser_eff < 30 * 86400 and not no_store:
        flag("HASHED_SHORT_TTL", f"content-hashed file cached only {fmt_s(browser_eff)}; it can be cached for a year with immutable")
    if "immutable" in cc and not hashed:
        flag("IMMUTABLE_UNHASHED", "immutable on a URL without a content hash; browsers will not pick up changes until max-age expires")
    if kind in ("asset", "image", "font") and (no_store or "no-cache" in cc) and not personal:
        flag("ASSET_NO_CACHING", "static file is never reused from cache; every page view downloads or revalidates it")
    if ("no-cache" in cc or browser_eff == 0) and not no_store and not validator and status == 200:
        flag("NO_VALIDATOR", "must revalidate but has no ETag or Last-Modified, so every check downloads the full body")
    if heuristic:
        flag("HEURISTIC_ONLY", f"no Cache-Control or Expires; caches guess {fmt_s(browser)} from Last-Modified")
    if "cache-control" not in h and expires is not None:
        flag("EXPIRES_ONLY", "only the legacy Expires header; add Cache-Control max-age")
    if "cache-control" not in h and "pragma" in h:
        flag("PRAGMA_ONLY", "Pragma: no-cache is a request header and has no effect on responses; use Cache-Control")
    if status >= 400 and max(browser_eff, shared_eff) > 300:
        flag("ERROR_CACHED", f"status {status} cached for {fmt_s(max(browser_eff, shared_eff))}; a brief error becomes a long outage")
    rec_kind = "personal" if personal and kind in ("api", "html", "other") else kind
    return {"label": block["label"], "status": status, "kind": kind, "hashed": hashed,
            "cache_control": h.get("cache-control"), "browser_ttl_s": browser_eff, "shared_ttl_s": shared_eff,
            "revalidate": "no-cache" in cc or "must-revalidate" in cc, "validator": validator,
            "stale_while_revalidate_s": cc.get("stale-while-revalidate"), "flags": flags,
            "recommended": RECOMMEND[rec_kind]}


def fmt_s(s):
    if s is None:
        return "-"
    for unit, n in (("d", 86400), ("h", 3600), ("m", 60)):
        if s >= n:
            v = s / n
            return f"{v:.0f}{unit}" if v == int(v) else f"{v:.1f}{unit}"
    return f"{s}s"


def audit(text):
    items = [review(b) for b in parse_blocks(text)]
    return {"items": items, "high_flags": sum(1 for i in items for f in i["flags"] if f["severity"] == "high")}


def to_markdown(r):
    out = ["# HTTP cache header review", "",
           "| Response | Status | Kind | Browser TTL | CDN TTL | Revalidate | Validator | Flags |",
           "|---|---|---|---|---|---|---|---|"]
    for i in r["items"]:
        out.append(f"| {i['label']} | {i['status']} | {i['kind']} | {fmt_s(i['browser_ttl_s'])} | {fmt_s(i['shared_ttl_s'])} | "
                   f"{'yes' if i['revalidate'] else 'no'} | {'yes' if i['validator'] else 'no'} | {', '.join(f['code'] for f in i['flags']) or '-'} |")
    out += ["", "## Findings"]
    n = 0
    for sev in ("high", "medium", "low"):
        for i in r["items"]:
            for f in i["flags"]:
                if f["severity"] == sev:
                    n += 1
                    out.append(f"- **{sev.upper()} {f['code']}** {i['label']}: {f['detail']}")
    if not n:
        out.append("- No findings.")
    out += ["", "## Suggested Cache-Control"]
    for i in r["items"]:
        if i["flags"]:
            out.append(f"- {i['label']}: `{i['recommended']}` (now: `{i['cache_control'] or 'none'}`)")
    return "\n".join(out)


def self_test():
    cases = []

    def ok(name, cond):
        cases.append((name, bool(cond)))

    def one(url, *lines):
        return review(parse_blocks("### GET " + url + "\nHTTP/1.1 200 OK\n" + "\n".join(lines))[0])

    def codes(item):
        return {f["code"] for f in item["flags"]}

    ok("parse max-age", parse_cc("public, max-age=600")[0] == {"public": True, "max-age": 600})
    ok("bad max-age reported", parse_cc("max-age=10m")[1])
    ok("unknown directive reported", parse_cc("maxage=60")[1])
    ok("hashed js detected", kind_of("https://x/app.3f9a1c7b.js", "") == ("hashed-asset", True))
    ok("unhashed css", kind_of("https://x/styles.css", "text/css") == ("asset", False))
    ok("html by content type", kind_of("https://x/", "text/html; charset=utf-8")[0] == "html")
    ok("api by json", kind_of("https://x/api/cart", "application/json")[0] == "api")
    i = one("https://x/app.3f9a1c7b.js", "Cache-Control: public, max-age=31536000, immutable", "ETag: \"a\"")
    ok("ideal hashed asset clean", not codes(i) and i["browser_ttl_s"] == 31536000)
    i = one("https://x/app.3f9a1c7b.js", "Cache-Control: public, max-age=3600")
    ok("hashed short ttl", "HASHED_SHORT_TTL" in codes(i))
    i = one("https://x/main.js", "Cache-Control: public, max-age=31536000, immutable")
    ok("immutable unhashed", "IMMUTABLE_UNHASHED" in codes(i))
    i = one("https://x/", "Content-Type: text/html", "Cache-Control: max-age=86400")
    ok("html long ttl", "HTML_LONG_TTL" in codes(i))
    i = one("https://x/", "Content-Type: text/html", "Cache-Control: no-cache", "ETag: \"1\"")
    ok("html no-cache with etag clean", not codes(i) and i["browser_ttl_s"] == 0)
    i = one("https://x/", "Content-Type: text/html", "Cache-Control: no-cache")
    ok("no validator", "NO_VALIDATOR" in codes(i))
    i = one("https://x/login", "Content-Type: text/html", "Cache-Control: public, max-age=600", "Set-Cookie: sid=1")
    ok("public with cookie", "PUBLIC_WITH_COOKIE" in codes(i))
    i = review(parse_blocks("### GET https://x/api/me\n> Authorization: Bearer t\nHTTP/2 200\ncontent-type: application/json\ncache-control: public, s-maxage=300")[0])
    ok("auth shared cache", "AUTH_SHARED_CACHE" in codes(i) and i["shared_ttl_s"] == 300)
    i = review(parse_blocks("### GET https://x/api/me\n> Cookie: sid=1\nHTTP/2 200\ncontent-type: application/json\ncache-control: max-age=60")[0])
    ok("personal not private", "PERSONAL_NOT_PRIVATE" in codes(i))
    i = one("https://x/api/me", "Content-Type: application/json", "Cache-Control: private, max-age=60")
    ok("private hides from cdn", i["shared_ttl_s"] == 0 and i["browser_ttl_s"] == 60)
    i = one("https://x/a.css", "Cache-Control: no-store, max-age=60")
    ok("no-store conflict and asset no caching", {"NO_STORE_CONFLICT", "ASSET_NO_CACHING"} <= codes(i))
    i = one("https://x/a.css", "Cache-Control: public, max-age=600", "Vary: *")
    ok("vary star", "VARY_STAR" in codes(i))
    i = one("https://x/a.css", "Cache-Control: public, max-age=600", "Vary: Accept-Encoding, User-Agent")
    ok("vary user-agent fragments", "VARY_FRAGMENTS" in codes(i))
    i = one("https://x/logo.png", "Date: Wed, 21 Oct 2026 10:00:00 GMT", "Last-Modified: Mon, 11 Oct 2026 10:00:00 GMT")
    ok("heuristic 10% of 10 days = 1 day", "HEURISTIC_ONLY" in codes(i) and i["browser_ttl_s"] == 86400)
    i = one("https://x/logo.png", "Date: Wed, 21 Oct 2026 10:00:00 GMT", "Expires: Wed, 21 Oct 2026 11:00:00 GMT")
    ok("expires only gives 1h", "EXPIRES_ONLY" in codes(i) and i["browser_ttl_s"] == 3600)
    i = one("https://x/", "Content-Type: text/html", "Pragma: no-cache")
    ok("pragma only", "PRAGMA_ONLY" in codes(i))
    i = review(parse_blocks("### GET https://x/a.css\nHTTP/1.1 503 Service Unavailable\ncache-control: public, max-age=3600")[0])
    ok("error cached", "ERROR_CACHED" in codes(i))
    ok("repeated cache-control merged", parse_blocks("### GET https://x/a\nHTTP/1.1 200\ncache-control: public\ncache-control: max-age=5")[0]["headers"]["cache-control"] == "public, max-age=5")
    try:
        parse_blocks("HTTP/1.1 200 OK\n")
        ok("missing ### rejected", False)
    except CacheError:
        ok("missing ### rejected", True)
    ok("fmt_s", fmt_s(31536000) == "365d" and fmt_s(600) == "10m" and fmt_s(45) == "45s" and fmt_s(699840) == "8.1d")
    passed = sum(c for _, c in cases)
    for name, c in cases:
        print(("PASS " if c else "FAIL ") + name)
    print(f"{passed}/{len(cases)} passed")
    return 0 if passed == len(cases) else 1


def main(argv=None):
    ap = argparse.ArgumentParser(description="Review HTTP caching headers.")
    ap.add_argument("file", nargs="?", help="responses text file ('-' for stdin)")
    ap.add_argument("--json", action="store_true")
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test:
        return self_test()
    if not a.file:
        ap.error("responses file required (or --self-test)")
    try:
        text = sys.stdin.read() if a.file == "-" else open(a.file, encoding="utf-8").read()
        r = audit(text)
    except (OSError, CacheError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(json.dumps(r, indent=2) if a.json else to_markdown(r))
    return 1 if r["high_flags"] else 0


if __name__ == "__main__":
    sys.exit(main())

# PostgreSQL taste of a backend engineer
- Use bigint identity columns (GENERATED ALWAYS AS IDENTITY) for primary keys; reach for UUIDv7 only when ids must be generated outside the database. Confidence: 0.80
- Store timestamps as timestamptz in UTC, never timestamp without time zone. Confidence: 0.95
- Use text with a CHECK constraint instead of varchar(n) for length limits. Confidence: 0.80
- Store money as numeric or as integer minor units, never as float. Confidence: 0.95
- Name tables in plural snake_case and foreign keys as <singular>_id. Confidence: 0.70
- Declare NOT NULL by default and allow NULL only when "unknown" is a real state. Confidence: 0.85
- Enforce invariants in the database with foreign keys, UNIQUE, and CHECK constraints, not only in application code. Confidence: 0.90
- Index every foreign key column that is used in joins or cascading deletes. Confidence: 0.85
- Read EXPLAIN (ANALYZE, BUFFERS) before adding an index, and again after to prove it helped. Confidence: 0.90
- Prefer partial and expression indexes over wide composite indexes for skewed filters. Confidence: 0.70
- Create indexes on large tables with CREATE INDEX CONCURRENTLY. Confidence: 0.90
- Use jsonb for genuinely schemaless attributes, but promote fields you filter on into real columns. Confidence: 0.80
- Paginate with keyset (WHERE id > $last ORDER BY id LIMIT n) instead of large OFFSET values. Confidence: 0.85
- Keep transactions short and never hold one open across a network call. Confidence: 0.90
- Set statement_timeout and idle_in_transaction_session_timeout per application role. Confidence: 0.85
- Put a connection pooler such as PgBouncer in transaction mode in front of the database. Confidence: 0.80
- Use SELECT ... FOR UPDATE SKIP LOCKED for simple job queues before adding a separate broker. Confidence: 0.70
- Use INSERT ... ON CONFLICT for idempotent upserts. Confidence: 0.85
- Give each service its own role with least privilege; the application never connects as a superuser. Confidence: 0.90
- Turn on pg_stat_statements and review the slowest queries every week. Confidence: 0.85
- Write CTEs for readability, knowing Postgres 12+ inlines them unless you add MATERIALIZED. Confidence: 0.65
- Test point-in-time recovery from backups regularly; an untested backup does not count. Confidence: 0.95

# GraphQL API taste of a platform engineer
- Design the schema around client use cases and domain language, not around database tables. Confidence: 0.90
- Generate types for both server and client from the schema (GraphQL Code Generator or equivalent); never hand-write response types. Confidence: 0.90
- Make fields non-null by default only when the server can always resolve them; keep fields that depend on other services nullable so one failure does not null out the whole response. Confidence: 0.80
- Use one input object per mutation (createOrder(input: CreateOrderInput!)) and return a payload type, not the bare entity. Confidence: 0.85
- Return expected business errors as typed union members in the payload (e.g. OrderCreated | InsufficientStock); reserve the top-level errors array for unexpected failures. Confidence: 0.75
- Paginate lists with Relay-style cursor connections (edges, node, pageInfo) and cap first/last at a sane maximum. Confidence: 0.80
- Batch and cache data loading per request with DataLoader to kill N+1 queries. Confidence: 0.95
- Enforce query depth and complexity limits at the gateway. Confidence: 0.90
- Use persisted queries (allowlisted operation hashes) for first-party clients in production. Confidence: 0.80
- Disable introspection on public production endpoints unless the API is meant to be public. Confidence: 0.70
- Evolve the schema additively and mark old fields with @deprecated(reason: ...); never version the endpoint with /v2. Confidence: 0.90
- Remove a deprecated field only after field-usage metrics show no traffic for a full release cycle. Confidence: 0.85
- Run schema diff checks in CI and fail the build on breaking changes. Confidence: 0.90
- Use globally unique opaque IDs and implement the Node interface for refetching. Confidence: 0.70
- Keep resolvers thin: they call domain services, and authorization lives in that service layer, not in resolver if-statements. Confidence: 0.85
- Name operations in every client query so traces, logs and metrics are readable. Confidence: 0.90
- Colocate fragments with the UI components that use them. Confidence: 0.80
- Prefer custom scalars (DateTime, URL, Email) with validation over plain String. Confidence: 0.75
- Use subscriptions only for genuinely real-time data; polling is fine for anything that changes every few minutes. Confidence: 0.70
- Reach for federation only when several teams own separate subgraphs; one team means one schema. Confidence: 0.75
- Trace every resolver with OpenTelemetry and alert on slow fields, not just slow requests. Confidence: 0.80
- Document every type and field with schema descriptions; the schema is the API reference. Confidence: 0.85

# Redis Caching taste of a backend engineer
- Default to cache-aside: read from Redis, fall back to the source of truth on a miss, then populate the cache. Confidence: 0.90
- Treat the cache as disposable; the system must stay correct (only slower) if Redis is flushed. Confidence: 0.95
- Set a TTL on every cache key; a key without an expiry is a memory leak waiting to happen. Confidence: 0.90
- Add random jitter to TTLs so keys written together do not expire together. Confidence: 0.85
- Namespace keys as service:entity:id:version (e.g. catalog:product:42:v3) and bump the version on breaking format changes. Confidence: 0.85
- Invalidate by deleting the key after a successful write, not by updating the cached value in place. Confidence: 0.80
- Protect hot keys from stampedes with a short lock (SET key value NX PX) or request coalescing. Confidence: 0.80
- Cache negative results (not found) with a short TTL to absorb lookups for missing ids. Confidence: 0.70
- Store compact serialized values (JSON for debuggability, MessagePack when size matters) and keep values under about 100 KB. Confidence: 0.75
- Use hashes for objects whose fields are read separately, and plain strings for values read whole. Confidence: 0.70
- Never run KEYS in production; use SCAN with a cursor. Confidence: 0.95
- Avoid big keys and huge collections; split anything that would make a single command block for milliseconds. Confidence: 0.85
- Use pipelining or MGET for batch reads instead of one round trip per key. Confidence: 0.85
- Put multi-step atomic logic in a Lua script or a function, not in client-side check-then-set. Confidence: 0.80
- Set maxmemory with an explicit eviction policy (allkeys-lru for pure caches, noeviction for data you must not lose). Confidence: 0.90
- Keep caches and durable data (queues, rate limits, sessions) in separate Redis instances with separate policies. Confidence: 0.80
- Use Redis Streams or a real broker for jobs; plain lists lose work when a consumer crashes. Confidence: 0.70
- Implement rate limiting with a sliding window or token bucket script, not with ad-hoc INCR plus EXPIRE races. Confidence: 0.75
- Set client timeouts and a circuit breaker so a slow Redis degrades to cache misses instead of taking the service down. Confidence: 0.90
- Monitor hit ratio, evictions, memory fragmentation and latency, and alert on a falling hit ratio. Confidence: 0.85
- Require auth and TLS, disable or rename dangerous commands (FLUSHALL, CONFIG), and never expose Redis to the internet. Confidence: 0.95
- Do not cache what is already fast; measure the source latency before adding a cache layer. Confidence: 0.80

# Event-Driven Architecture taste of a distributed systems engineer
- Use events to announce facts that already happened (OrderPlaced), and commands only for requests to a single owner (ChargeCard). Confidence: 0.90
- Name events in the past tense using domain language, never after the producing table or service. Confidence: 0.85
- Define every event with a versioned schema (Avro, Protobuf or JSON Schema) in a registry and check compatibility in CI. Confidence: 0.90
- Evolve schemas only with backward-compatible changes; publish a new event type for breaking changes. Confidence: 0.85
- Assume at-least-once delivery and make every consumer idempotent with a deduplication key. Confidence: 0.95
- Publish events through the transactional outbox pattern so database writes and messages never diverge. Confidence: 0.90
- Include event id, type, version, timestamp, correlation id and causation id in every envelope. Confidence: 0.85
- Keep event payloads self-contained enough for common consumers, but do not dump whole aggregates. Confidence: 0.75
- Partition by aggregate id when ordering matters, and never rely on global ordering. Confidence: 0.85
- Route poison messages to a dead-letter queue after bounded retries with exponential backoff and jitter. Confidence: 0.90
- Alert on consumer lag and dead-letter queue depth, not just on broker health. Confidence: 0.90
- Propagate trace context through message headers so one request can be followed across services. Confidence: 0.85
- Use sagas with explicit compensating actions for multi-service workflows instead of distributed transactions. Confidence: 0.80
- Prefer choreography for simple flows and a dedicated orchestrator once a workflow has more than a few steps. Confidence: 0.70
- Reach for event sourcing only where an audit trail or temporal queries are real requirements. Confidence: 0.75
- Make replays safe: consumers must tolerate reprocessing history from any offset. Confidence: 0.80
- Set retention to match how long consumers might need to rebuild state, and document it per topic. Confidence: 0.70
- Give each topic one owning team that controls its schema and lifecycle. Confidence: 0.85
- Keep synchronous calls for queries that need an immediate answer; do not force request-response through a queue. Confidence: 0.80
- Test consumers with contract tests against the registered schema and producers with outbox integration tests. Confidence: 0.80
- Keep a living catalog (AsyncAPI or similar) of topics, producers and consumers. Confidence: 0.75

# Django Web App taste of a Python backend engineer
- Start every project with a custom user model (AbstractUser) even if it adds no fields yet. Confidence: 0.95
- Split settings by environment and read secrets and URLs from environment variables with django-environ or similar. Confidence: 0.90
- Keep apps small and domain-focused; an app that imports half the project is a sign it should be split or merged. Confidence: 0.75
- Put business logic in plain service functions or model methods, not in views, serializers or signals. Confidence: 0.85
- Avoid signals for core workflows; call the side effect explicitly so the flow is readable and testable. Confidence: 0.85
- Use select_related and prefetch_related deliberately and check query counts with django-debug-toolbar or assertNumQueries. Confidence: 0.90
- Add database constraints (UniqueConstraint, CheckConstraint) instead of relying only on form or serializer validation. Confidence: 0.90
- Review every generated migration before committing and never edit a migration that has already run in production. Confidence: 0.90
- Write data migrations with RunPython using apps.get_model, never importing models directly. Confidence: 0.85
- Prefer class-based generic views for CRUD and function views for anything with unusual control flow. Confidence: 0.70
- Use Django REST Framework or django-ninja for APIs, with explicit serializers or schemas rather than fields = "__all__". Confidence: 0.85
- Wrap multi-step writes in transaction.atomic and enqueue background jobs with transaction.on_commit. Confidence: 0.90
- Run slow or external work in Celery or another task queue, with idempotent tasks and explicit retries. Confidence: 0.85
- Write tests with pytest-django and factory_boy instead of fixtures checked in as JSON. Confidence: 0.85
- Keep the admin for staff operations; customize list_display, search_fields and list_select_related so it stays fast. Confidence: 0.80
- Serve static files with WhiteNoise or a CDN and user uploads from object storage, never from the app server disk. Confidence: 0.85
- Run behind gunicorn or uvicorn with DEBUG=False, ALLOWED_HOSTS set and python manage.py check --deploy passing in CI. Confidence: 0.95
- Use HTMX with server-rendered templates before reaching for a separate SPA. Confidence: 0.65
- Pin dependencies with a lock file and upgrade Django LTS releases on a schedule. Confidence: 0.80
- Lint and format with Ruff and type check with mypy plus django-stubs. Confidence: 0.75

# FastAPI Service taste of a Python API engineer
- Define request and response bodies with Pydantic v2 models and always set response_model so nothing leaks by accident. Confidence: 0.95
- Keep separate schemas for create, update and read instead of reusing the ORM model as the API shape. Confidence: 0.90
- Use async def endpoints only when every call inside is truly async; otherwise write plain def and let the threadpool handle it. Confidence: 0.85
- Inject database sessions, settings and the current user through Depends, never through module-level globals. Confidence: 0.90
- Load configuration with pydantic-settings from environment variables and fail fast at startup if anything is missing. Confidence: 0.90
- Split routes into APIRouter modules by domain and keep main.py limited to app creation and wiring. Confidence: 0.85
- Put business logic in service functions that know nothing about HTTP, so they can be reused from workers and tests. Confidence: 0.85
- Use SQLAlchemy 2.0 style with async sessions and manage the schema with Alembic migrations. Confidence: 0.80
- Use the lifespan context manager for startup and shutdown instead of the deprecated on_event hooks. Confidence: 0.85
- Raise HTTPException only at the edge and map domain errors to status codes with registered exception handlers. Confidence: 0.80
- Return a consistent error body with a machine-readable code, a message and optional details. Confidence: 0.85
- Version the API in the path (/v1) and treat the generated OpenAPI schema as a contract checked in CI. Confidence: 0.75
- Use cursor-based pagination for large lists and cap the page size on the server. Confidence: 0.80
- Test with pytest and httpx.AsyncClient, overriding dependencies instead of patching internals. Confidence: 0.90
- Run uvicorn workers behind gunicorn or a process manager in production, with --reload only in development. Confidence: 0.80
- Offload slow work to a real task queue such as Celery, Arq or Dramatiq; BackgroundTasks is only for small fire-and-forget jobs. Confidence: 0.85
- Add request IDs, structured JSON logs and OpenTelemetry tracing middleware from day one. Confidence: 0.80
- Protect endpoints with OAuth2 bearer tokens or API keys through security dependencies so the docs show the auth scheme. Confidence: 0.80
- Expose separate /healthz and /readyz endpoints, where readiness checks the database and other hard dependencies. Confidence: 0.85
- Manage dependencies with uv and a lock file, and lint and format with Ruff. Confidence: 0.75
- Type check with mypy or pyright in strict mode on the service package. Confidence: 0.70

# Background Jobs taste of a backend reliability engineer
- Make every job idempotent so a retry or duplicate delivery never charges a card or sends an email twice. Confidence: 0.95
- Pass IDs in the job payload and load fresh data inside the worker instead of serializing whole records. Confidence: 0.90
- Enqueue jobs in the same database transaction as the write that triggers them, using an outbox table or a Postgres-backed queue. Confidence: 0.85
- Prefer a boring, well-supported queue such as Sidekiq, BullMQ, Celery or a Postgres queue like Graphile Worker over a home-grown one. Confidence: 0.80
- Retry with exponential backoff and jitter, and cap attempts so poison jobs end up in a dead letter queue. Confidence: 0.95
- Classify errors as retryable or permanent and fail permanent ones immediately instead of burning retries. Confidence: 0.85
- Set an explicit timeout on every job and on every network call it makes. Confidence: 0.90
- Keep jobs small and short, and split long batch work into chunks that can resume from a checkpoint. Confidence: 0.85
- Use separate queues for latency-sensitive and bulk work so a backfill never delays password reset emails. Confidence: 0.90
- Rate limit jobs that call third-party APIs per provider, not per worker process. Confidence: 0.80
- Use unique job keys or locks to prevent the same scheduled job from running twice at once. Confidence: 0.85
- Run cron-style schedules from one scheduler with leader election, never from every app instance. Confidence: 0.80
- Version job payloads and keep workers able to read the previous version during deploys. Confidence: 0.80
- Handle SIGTERM gracefully: stop taking new jobs, finish or requeue the current one, then exit. Confidence: 0.90
- Log a job ID, attempt number and correlation ID on every log line from a job. Confidence: 0.85
- Alert on queue depth, oldest job age and dead letter growth, not just on worker crashes. Confidence: 0.90
- Give the dead letter queue an owner and a documented replay command. Confidence: 0.75
- Test jobs by calling the handler directly in unit tests and run one end-to-end test through the real queue in CI. Confidence: 0.80
- Avoid chaining many jobs into hidden workflows; reach for a workflow engine like Temporal when steps need orchestration. Confidence: 0.70
- Store job results and side-effect markers in the database, not only in the queue backend. Confidence: 0.75

# Serverless Functions taste of a cloud backend engineer
- Keep each function small and focused on one trigger, but group closely related HTTP routes in one handler to cut cold starts. Confidence: 0.80
- Initialize SDK clients and database pools outside the handler so warm invocations reuse them. Confidence: 0.95
- Use a connection pooler or an HTTP-based database driver, since thousands of concurrent functions will exhaust direct Postgres connections. Confidence: 0.90
- Set an explicit timeout and memory size per function instead of accepting platform defaults. Confidence: 0.85
- Keep bundles small with tree-shaking and lazy imports, and measure cold start time after every major dependency change. Confidence: 0.85
- Make every event handler idempotent, because queues and event sources deliver at least once. Confidence: 0.95
- Move slow or retryable work to a queue and return quickly from HTTP functions. Confidence: 0.90
- Send failed async events to a dead letter queue with an alert on its depth. Confidence: 0.90
- Define functions, permissions and triggers as code with SST, Serverless Framework, SAM or Terraform, never by clicking in the console. Confidence: 0.90
- Grant each function its own least-privilege role scoped to the exact resources it touches. Confidence: 0.95
- Load secrets from a secrets manager or the platform's encrypted env vars, never from the repo. Confidence: 0.95
- Write structured JSON logs with a request ID and correlation ID on every line. Confidence: 0.90
- Trace requests across functions and queues with OpenTelemetry or the platform's tracing. Confidence: 0.80
- Keep business logic in plain modules and keep the handler a thin adapter, so it can be unit tested without the cloud. Confidence: 0.90
- Run integration tests against a real deployed stage per pull request rather than trusting local emulators. Confidence: 0.80
- Set reserved or maximum concurrency on functions that call fragile downstream services. Confidence: 0.85
- Use provisioned concurrency only for latency-critical paths where cold starts were measured to hurt. Confidence: 0.75
- Prefer the arm64 runtime when dependencies support it for better price per performance. Confidence: 0.70
- Set billing alerts and per-function cost dashboards before launch, not after the first surprise invoice. Confidence: 0.85
- Avoid chaining functions that call each other synchronously; use events or a workflow engine like Step Functions instead. Confidence: 0.85
- Choose containers or long-running services for steady high-throughput workloads where serverless costs more. Confidence: 0.75

# Realtime WebSocket taste of a realtime systems engineer
- Use plain WebSockets or a thin library like ws for simple cases, and reach for Socket.IO only when you need its fallbacks and rooms. Confidence: 0.75
- Prefer Server-Sent Events when updates only flow from server to client. Confidence: 0.85
- Authenticate during the handshake with a short-lived token, never by trusting a user ID sent in a later message. Confidence: 0.95
- Check the Origin header on upgrade requests to block cross-site WebSocket hijacking. Confidence: 0.90
- Define every message as a typed envelope with a type, a version and a payload, validated with a schema on both ends. Confidence: 0.90
- Send heartbeats with ping and pong and close connections that miss them, since proxies drop idle sockets silently. Confidence: 0.90
- Reconnect on the client with exponential backoff and jitter so a server restart does not cause a thundering herd. Confidence: 0.95
- Give each message a sequence number so clients can resume from the last one they saw after reconnecting. Confidence: 0.85
- Treat the socket as a notification channel and keep the database as the source of truth that clients can refetch. Confidence: 0.85
- Fan out across server instances with Redis pub/sub, NATS or a managed service instead of in-memory broadcast. Confidence: 0.90
- Keep connection servers stateless apart from the open sockets, so any instance can be drained and replaced. Confidence: 0.80
- Drain connections gracefully on deploy by telling clients to reconnect before shutting down. Confidence: 0.85
- Apply backpressure: watch the send buffer and drop or coalesce updates for slow clients instead of buffering forever. Confidence: 0.85
- Rate limit incoming messages per connection and cap message size. Confidence: 0.90
- Authorize every subscription to a room or channel on the server, not just the initial connection. Confidence: 0.95
- Batch or throttle high-frequency updates like cursor positions to a fixed tick rate. Confidence: 0.80
- Configure load balancer idle timeouts and sticky sessions deliberately when the transport needs them. Confidence: 0.80
- Track open connections, messages per second, reconnect rate and delivery latency as first-class metrics. Confidence: 0.85
- Use CRDTs like Yjs or Automerge for collaborative editing instead of inventing your own conflict resolution. Confidence: 0.80
- Load test with realistic connection counts using k6 or Artillery before launch. Confidence: 0.80