diff --git a/CLAUDE.md b/CLAUDE.md index 8246b96..a861563 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,19 +15,40 @@ Recipes live in the `Justfile` (`just --list`); the bare `just` runs the full The CI `DB_DSN` format: `postgresql+asyncpg://postgres:postgres@localhost:5432/postgres` -## Architecture - -The package (`db_retry/`) exposes five public symbols via `__init__.py`: - -- **`postgres_retry`** (`retry.py`) — async tenacity decorator that retries on `asyncpg.SerializationError` (40001) and `asyncpg.PostgresConnectionError` (08000/08003). Walks the outer `__cause__`/`__context__` chain to find any `DBAPIError`, then inspects `DBAPIError.orig.__cause__` to distinguish retriable errors from others like `StatementCompletionUnknownError` (40002). The chain walk lets retries fire when the `DBAPIError` is re-raised by a wrapper (e.g. advanced-alchemy's `wrap_sqlalchemy_exception()` surfacing it as `RepositoryError`/`IntegrityError`). Supports bare `@postgres_retry` (uses default) and `@postgres_retry(retries=N)` for per-callsite override. +## Workflow -- **`build_connection_factory`** (`connections.py`) — returns an async callable suitable for SQLAlchemy's `async_engine_from_config`. Handles multi-host DSNs by randomizing host order (load balancing) and attempting all hosts on timeout before raising `TargetServerAttributeNotMatched`. +Planning follows [`planning/README.md`](planning/README.md) — its **Quick path** +is the authoritative convention for making a change (choose a lane, create a +bundle under `planning/changes/`, ship the `architecture/` promotion in the same +PR). Run `just check-planning` (also wired into `just lint-ci`) before pushing. -- **`build_db_dsn`** / **`is_dsn_multihost`** (`dsn.py`) — parse and construct `sqlalchemy.URL` objects. Multi-host DSNs encode additional hosts in query parameters. Existing `target_session_attrs` in the DSN is preserved (not overwritten). - -- **`Transaction`** (`transaction.py`) — frozen dataclass context manager wrapping `AsyncSession`. Supports optional isolation level (e.g., `"SERIALIZABLE"`). Auto-rolls back on `__aexit__` if the session is still in a transaction (i.e. no explicit `.commit()` or `.rollback()` was called). Uses `typing.Self` (no `typing_extensions` dependency). +## Architecture -- **`settings.py`** — exposes `get_retries_number()` which reads `DB_RETRY_RETRIES_NUMBER` env var at call time (default: 3), allowing `monkeypatch.setenv` to work in tests. +> Quick orientation only. The authoritative, code-current account of each +> capability lives in [`architecture/`](architecture/) — one file per +> capability. **When a change alters a capability's behavior, update the matching +> `architecture/.md` in the same PR** — that promotion is what keeps +> `architecture/` true; code that changes without it silently rots the truth home. + +The package (`db_retry/`) exposes five public symbols via `__init__.py`. Read +the matching capability file before changing behavior: + +| Symbol(s) | Source | Capability file | +|---|---|---| +| `postgres_retry` | `retry.py` | [architecture/retry.md](architecture/retry.md) | +| `build_connection_factory` | `connections.py` | [architecture/connections.md](architecture/connections.md) | +| `build_db_dsn`, `is_dsn_multihost` | `dsn.py` | [architecture/dsn.md](architecture/dsn.md) | +| `Transaction` | `transaction.py` | [architecture/transaction.md](architecture/transaction.md) | +| `get_retries_number` | `settings.py` | [architecture/settings.md](architecture/settings.md) | + +- **`postgres_retry`** is an async tenacity decorator that retries on + `asyncpg.SerializationError` (40001) and `asyncpg.PostgresConnectionError` + (08xxx), walking the `__cause__`/`__context__` chain to find a retriable + `DBAPIError` even when re-wrapped. Bare `@postgres_retry` or + `@postgres_retry(retries=N)`. +- **`build_connection_factory`** returns an async creator for + `async_engine_from_config`, load-balancing and failing over across multi-host + DSNs before raising `TargetServerAttributeNotMatched`. ## Linting / Type Checking diff --git a/Justfile b/Justfile index b446304..50b6fed 100644 --- a/Justfile +++ b/Justfile @@ -27,6 +27,13 @@ lint-ci: uv run ruff format --check uv run ruff check --no-fix uv run ty check + uv run python planning/index.py --check + +index: + uv run python planning/index.py + +check-planning: + uv run python planning/index.py --check publish: rm -rf dist diff --git a/architecture/README.md b/architecture/README.md new file mode 100644 index 0000000..90bd737 --- /dev/null +++ b/architecture/README.md @@ -0,0 +1,26 @@ +# Architecture + +The living truth about what `db-retry` does **now** — one file per capability, +updated by hand whenever a change ships. The *why* and *how it got here* live in +[`../planning/changes/`](../planning/changes/) — and decisions deliberately taken, +including options rejected, in [`../planning/decisions/`](../planning/decisions/); +this directory is the present. + +These files carry **no frontmatter** — they are prose, dated by git. + +## Capabilities + +- [retry.md](retry.md) — `postgres_retry`, the async tenacity decorator and its + cause-chain retry predicate. +- [connections.md](connections.md) — `build_connection_factory`, multi-host + load balancing and failover. +- [dsn.md](dsn.md) — `build_db_dsn` / `is_dsn_multihost`, DSN parsing and + construction. +- [transaction.md](transaction.md) — `Transaction`, the session context manager. +- [settings.md](settings.md) — `get_retries_number`, the env-driven retry count. + +## Promotion rule + +Shipping a change hand-edits the affected capability file(s) here to match the +new reality, in the same PR as the code. The change bundle stays in place under +[`../planning/changes/`](../planning/changes/) — no folder move. diff --git a/architecture/connections.md b/architecture/connections.md new file mode 100644 index 0000000..0e6cb28 --- /dev/null +++ b/architecture/connections.md @@ -0,0 +1,52 @@ +# Connections + +`build_connection_factory` (in `db_retry/connections.py`) returns an async +callable `() -> asyncpg.Connection` suitable for SQLAlchemy's +`async_engine_from_config` (the `async_creator` hook). Its job is to connect to +PostgreSQL across a multi-host DSN with load balancing and per-host failover. + +## Signature + +```python +build_connection_factory(url: sqlalchemy.URL, timeout: float) + -> Callable[[], Awaitable[asyncpg.Connection]] +``` + +The `url` is translated into asyncpg connect args **once**, at factory-build +time, via `PGDialect_asyncpg().create_connect_args(url)`. `target_session_attrs` +is popped from those args and, if present, wrapped in asyncpg's +`SessionAttribute` enum — so a `target_session_attrs` carried on the DSN (e.g. +`read-write`/`prefer-standby` set by [`build_db_dsn`](dsn.md)) **is honored**, +not discarded. + +## Host handling + +`host` and `port` are popped from the connect args: + +- **Multi-host** (both are lists): they are zipped into `(host, port)` pairs + (`strict=True` — lengths must match), the pair list is `random.shuffle`d for + load balancing, then split back into parallel `hosts`/`ports` lists. The + shuffled pair list is also retained for the failover path. +- **Single-host** (scalars): used as-is; the retained pair list is empty. + +## Connect and failover + +The returned `_connection_factory`: + +1. Attempts one `asyncpg.connect(...)` against the full host/port set with the + given `timeout` and `target_session_attrs`. On success, returns immediately. +2. On `TimeoutError`, if there is no multi-host pair list it re-raises. With a + pair list, it logs a warning and falls through to host-by-host probing. +3. Re-shuffles a copy of the pairs and tries each `(host, port)` individually, + swallowing `TimeoutError`, `OSError`, and `asyncpg.TargetServerAttributeNotMatched` + (logging a warning per failed host) and returning the first that connects. +4. If every host fails, raises `asyncpg.TargetServerAttributeNotMatched` naming + the unmet `target_session_attrs`. + +The two-stage design lets the fast path use asyncpg's own multi-host attempt, +and only pays the per-host cost when the bulk attempt times out — typically when +`target_session_attrs` (e.g. `read-write`) excludes some hosts. + +## Related + +- [dsn.md](dsn.md) — how multi-host DSNs and `target_session_attrs` are encoded. diff --git a/architecture/dsn.md b/architecture/dsn.md new file mode 100644 index 0000000..fd10931 --- /dev/null +++ b/architecture/dsn.md @@ -0,0 +1,47 @@ +# DSN + +`db_retry/dsn.py` parses and constructs `sqlalchemy.URL` objects for +multi-host PostgreSQL DSNs. Two public functions. + +## `build_db_dsn` + +```python +build_db_dsn( + db_dsn: str, + database_name: str, + use_replica: bool = False, + drivername: str = "postgresql", +) -> sqlalchemy.URL +``` + +Takes a stored DSN and returns a new `URL` with three things replaced: + +- **`database`** ← `database_name`. The stored DSN carries a placeholder + database (the maintained format is + `postgresql://login:password@/db_placeholder?host=host1&host=host2` — empty + host in the authority, real hosts in repeated `host` query params, per + SQLAlchemy's [multiple-fallback-hosts](https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#specifying-multiple-fallback-hosts) + form); the real service database name is substituted here. +- **`drivername`** ← `drivername` (default `postgresql`; callers pass + `postgresql+asyncpg` to get the async dialect). +- **`target_session_attrs`** ← `prefer-standby` when `use_replica` else + `read-write`. This is a dict union (`existing_query | {target_session_attrs: …}`), + so the computed value **overwrites** any `target_session_attrs` already on the + DSN — `use_replica` is authoritative. (The other existing query params are + preserved.) Note: honoring a *pre-existing* `target_session_attrs` happens + downstream in [`build_connection_factory`](connections.md), not here. + +## `is_dsn_multihost` + +```python +is_dsn_multihost(db_dsn: str) -> bool +``` + +`True` when the DSN's `host` query param is a tuple of length > 1 — i.e. the +multiple-fallback-hosts form with at least two hosts. A single `host` param or a +host in the authority (`@host/db`) is **not** multi-host. + +## Related + +- [connections.md](connections.md) — consumes the constructed `URL` and the + multi-host encoding. diff --git a/architecture/retry.md b/architecture/retry.md new file mode 100644 index 0000000..5969954 --- /dev/null +++ b/architecture/retry.md @@ -0,0 +1,59 @@ +# Retry + +`postgres_retry` (in `db_retry/retry.py`) is an async decorator that retries a +coroutine function when PostgreSQL raises a transient error — a serialization +failure or a lost connection — and gives up on everything else. + +## Public surface + +```python +@postgres_retry # bare — uses the default retry count +async def handler(...) -> ...: ... + +@postgres_retry(retries=5) # per-callsite override +async def handler(...) -> ...: ... +``` + +Two `typing.overload`s back the dual form: called with a function it returns the +wrapped function; called with `func=None` (i.e. `@postgres_retry(...)`) it +returns a decorator. The wrapped function keeps its signature via +`functools.wraps`. `retries` defaults to `None`, which defers to +[`settings.get_retries_number()`](settings.md) **at call time** — so the env var +is read per invocation, not frozen at decoration. + +## Retry engine + +Each call builds a `tenacity.AsyncRetrying` with: + +- `stop=stop_after_attempt(retries or get_retries_number())` +- `wait=wait_exponential_jitter()` — exponential backoff with jitter +- `retry=retry_if_exception(_retry_handler)` — the predicate below +- `reraise=True` — the **original** exception propagates after the last attempt, + not tenacity's `RetryError` +- `before=before_log(logger, DEBUG)` — debug log before each attempt + +## What counts as retriable + +`_is_retriable_dbapi_error` returns `True` only for a `sqlalchemy.exc.DBAPIError` +whose `.orig` is set and whose `.orig.__cause__` is an +`asyncpg.SerializationError` (SQLSTATE `40001`) or +`asyncpg.PostgresConnectionError` (class `08`, e.g. `08000`/`08003`). This +deliberately excludes lookalikes such as `StatementCompletionUnknownError` +(`40002`), where the statement's outcome is unknown and a blind retry is unsafe. + +## Cause-chain walk + +`_retry_handler` does not inspect only the raised exception — it walks the +`__cause__`/`__context__` chain (following `__cause__` first, then +`__context__`), guarding against cycles with a `seen` set of `id()`s, and +returns `True` as soon as any link is a retriable `DBAPIError`. + +The walk matters because the `DBAPIError` is often re-raised inside another +exception. For example advanced-alchemy's `wrap_sqlalchemy_exception()` surfaces +it as `RepositoryError`/`IntegrityError` with the real `DBAPIError` hanging off +`__cause__`; the walk lets the retry still fire. Both retry and give-up paths +emit a debug log. + +## Related + +- [settings.md](settings.md) — where the default attempt count comes from. diff --git a/architecture/settings.md b/architecture/settings.md new file mode 100644 index 0000000..99ac8cd --- /dev/null +++ b/architecture/settings.md @@ -0,0 +1,24 @@ +# Settings + +`db_retry/settings.py` holds the package's one piece of runtime configuration: +the default retry count. + +## `get_retries_number` + +```python +def get_retries_number() -> int: + return int(os.getenv("DB_RETRY_RETRIES_NUMBER", "3")) +``` + +Reads the `DB_RETRY_RETRIES_NUMBER` environment variable **at call time**, +defaulting to `3`. It is a function — not a module-level constant — precisely so +the value is re-read on every call: tests can `monkeypatch.setenv(...)` and see +the new value, and a deployment can change the env without re-importing. + +[`postgres_retry`](retry.md) calls this whenever its own `retries` argument is +`None`, so the env var sets the default attempt count while a per-callsite +`retries=N` overrides it. + +## Related + +- [retry.md](retry.md) — the sole consumer. diff --git a/architecture/transaction.md b/architecture/transaction.md new file mode 100644 index 0000000..0268c40 --- /dev/null +++ b/architecture/transaction.md @@ -0,0 +1,42 @@ +# Transaction + +`Transaction` (in `db_retry/transaction.py`) is an async context manager that +wraps a SQLAlchemy `AsyncSession`, opening a transaction on entry and cleaning +up on exit. + +## Shape + +```python +@dataclasses.dataclass(kw_only=True, frozen=True, slots=True) +class Transaction: + session: AsyncSession + isolation_level: IsolationLevel | None = None +``` + +A frozen, slotted, keyword-only dataclass: `Transaction(session=..., isolation_level=...)`. +`__aenter__` returns `typing.Self` (no `typing_extensions` dependency). + +## Entry + +`async with Transaction(session=s) as tx:` + +1. If `isolation_level` is set, calls `session.connection(execution_options={"isolation_level": ...})` + to apply it (e.g. `"SERIALIZABLE"`). +2. If the session is **not** already in a transaction, calls `session.begin()`. + An already-open transaction is adopted rather than nested. + +## Exit + +`__aexit__` does **not** suppress exceptions (returns `None`) and always runs, +on both the success and error path: + +1. If the session is still in a transaction, `session.rollback()`. This is the + auto-rollback: a block that neither `.commit()`s nor `.rollback()`s — or that + raises — leaves the transaction open, and exit rolls it back. A block that + already committed/rolled back is no longer in a transaction, so nothing is + undone. +2. `session.close()` — always, regardless of outcome. + +So commit is explicit: call `tx.commit()` inside the block to persist; otherwise +the work is rolled back. `tx.rollback()` is also exposed for an explicit early +rollback. Both delegate straight to the session. diff --git a/planning/.convention-version b/planning/.convention-version new file mode 100644 index 0000000..3eefcb9 --- /dev/null +++ b/planning/.convention-version @@ -0,0 +1 @@ +1.0.0 diff --git a/planning/README.md b/planning/README.md new file mode 100644 index 0000000..2fdd7ac --- /dev/null +++ b/planning/README.md @@ -0,0 +1,128 @@ +# Planning + +How change is planned and recorded in `db-retry`. Two axes, never mixed: +[`architecture/`](../architecture/) (repo root) holds the living truth about +what the package does **now**; [`changes/`](changes/) records how it got there. + +The convention below is the portable +[`lesnik512/planning-convention`](https://github.com/lesnik512/planning-convention) +(applied version in [`.convention-version`](.convention-version)); to update it, +run that repo's `APPLY.md` flow. The `## Index` and `## Other` sections are +repo-local. + +## Quick path (start here) + +> The fast lane for making a change. The full reference is in +> [Conventions](#conventions) below — read it only when this isn't enough. + +**1. Choose a lane — first matching rule wins:** + +1. Any of: needs design judgment · new file/module · public-API change · + cross-cutting or multi-file · non-trivial test design → **Full** + (`design.md` + `plan.md`) +2. Purely mechanical: typo · dep bump · linter/formatter/CI tweak · + mechanical rename · single-line config → **Tiny** (no bundle, conventional + commit) +3. Small-but-real, none of the above: ≲30 LOC net · ≤2 files · no new file · + no public-API change · one straightforward test → **Lightweight** + (`change.md`) + +Ambiguous between two? Take the heavier. A `change.md` that outgrows its lane +splits into `design.md` + `plan.md`. + +**2. Create the bundle** (Full / Lightweight only): +`planning/changes/YYYY-MM-DD.NN-/`, where `.NN` is a zero-padded +intra-day counter. Copy the matching template from +[`_templates/`](_templates/). + +**3. Ship in the implementing PR:** hand-edit the affected +`architecture/.md`, finalize the bundle's `summary:` to the +realized result, and run `just check-planning` before pushing. + +## Conventions + +> This is the portable convention, sourced from the canonical repo +> [`lesnik512/planning-convention`](https://github.com/lesnik512/planning-convention) +> (applied version in [`.convention-version`](.convention-version)). To update +> it, run that repo's `APPLY.md` flow. The generated change index (`just index`) +> and the `## Other` pointers below are repo-local. + +### Two axes, never mixed + +- **`architecture/` (repo root) — the present.** One file per capability, + living prose, updated in the same PR that ships the change. The truth home. +- **`planning/changes/` — the past-and-pending.** One folder per change, + kept in place after ship. + +A change **promotes** its conclusions into the affected +`architecture/.md` by hand **in the implementing PR, alongside the +code** — the edit rides in the same diff and is reviewed with it, never applied +as a separate post-merge step. That hand-edit is what keeps `architecture/` +true; the bundle stays in `changes/` as the *why*. + +### Change bundles + +A change is a folder `changes/YYYY-MM-DD.NN-/`: + +- `YYYY-MM-DD` — proposal date; `.NN` — zero-padded intra-day counter + (`.01`, `.02`, …) that breaks same-date ties so the timeline sorts stably. +- `` — kebab-case description, not a story ID. + +`summary` is written when the change is created (the intent one-liner) and +**finalized at ship** to state the realized result — set in the implementing +PR, alongside the code and the `architecture/` promotion. No post-merge +bookkeeping, no folder move. `date` and `slug` are never written — they are +read from the bundle's directory name. + +### Three lanes + +| Lane | Artifacts | Use when | +|------|-----------|----------| +| **Full** | `design.md` + `plan.md` | design judgment; new file/module; public-API change; cross-cutting/multi-file; non-trivial test design | +| **Lightweight** | `change.md` | small-but-real: ≲30 LOC net, ≤2 files, no new file, no public-API change, single straightforward test | +| **Tiny** | none — conventional commit | typo, dep bump, linter/formatter/CI tweak, mechanical rename, single-line config | + +Heavier lane wins on ambiguity. A `change.md` that outgrows its lane splits +into `design.md` + `plan.md`. + +### Artifacts at a glance + +- **`design.md`** — the spec: the *thinking* (why, design, trade-offs, scope). +- **`plan.md`** — the plan: the *sequencing* (the executor's task checklist). +- **`change.md`** — both, condensed, for the lightweight lane. +- **`releases/.md`** — per-release user-facing notes. +- **`audits/-.md`** — findings from a code/docs/bug-hunt sweep; + spawns fix changes. +- **`retros/-.md`** — what we learned after a body of work. +- **`deferred.md`** — real-but-unscheduled items, each with a revisit trigger. +- **`decisions/-.md`** — one file per design decision taken + (especially options *rejected*), each with a revisit trigger; listed by + `just index`. + +Templates live in [`_templates/`](_templates/). + +### Frontmatter + +`date` and `slug` are **derived from the directory / file name** — never +repeated in frontmatter. So: + +- `design.md` / `change.md`: `summary` (single line) only. +- `plan.md`: **no frontmatter** — its identity is the bundle directory. +- `decisions/*.md`: `status` (accepted|superseded), `summary`, and optional + `supersedes` / `superseded_by`. +- Files in `architecture/` carry **no** frontmatter — living prose, dated by git. + +**`summary`** is one line: written at creation as the intent, then **finalized +at ship** to state the realized result — what shipped and its effect. It is the +only field the index renders. + +## Index + +Run `just index` to print the change/decision index (newest-first) to stdout; +`just check-planning` validates bundle and decision shape (wired into +`just lint-ci`). + +## Other + +- [`architecture/`](../architecture/) — the truth home, one file per capability. +- [`deferred.md`](deferred.md) — real-but-unscheduled items with revisit triggers. diff --git a/planning/_templates/change.md b/planning/_templates/change.md new file mode 100644 index 0000000..d4c8962 --- /dev/null +++ b/planning/_templates/change.md @@ -0,0 +1,32 @@ +--- +summary: One line — shown in the generated index. Written at creation; finalize at ship to state the realized result. +--- + +# Change: One-line capitalized title + +**Lane:** lightweight — ≲30 LOC net, ≤2 files, no new file, no public-API +change, a single straightforward test. If it outgrows this, split into +`design.md` + `plan.md`. + +## Goal + +One or two sentences: what changes and why. + +## Approach + +The shape of the change in brief — enough that a reviewer sees the design +without a full spec. Link the truth home (`architecture/.md`) if a +capability contract moves. + +## Files + +- `path/to/file.py` — what changes +- `tests/test_x.py` — test added / updated + +## Verification + +- [ ] Failing test first — command + expected error. +- [ ] Apply the change. +- [ ] Test passes — command. +- [ ] `just test` — full suite green. +- [ ] `just lint` — clean. diff --git a/planning/_templates/decision.md b/planning/_templates/decision.md new file mode 100644 index 0000000..45ccaf0 --- /dev/null +++ b/planning/_templates/decision.md @@ -0,0 +1,23 @@ +--- +status: accepted # accepted | superseded +summary: One line — shown in `just index`. +supersedes: null +superseded_by: null +--- + +# One-line capitalized title + +**Decision:** What was decided, in a sentence. + +## Context + +Why this came up; the options that were on the table. + +## Decision & rationale + +The call and why — including why the alternatives were rejected. Enough that a +future explorer doesn't re-litigate it. + +## Revisit trigger + +The concrete signal that should reopen this decision. diff --git a/planning/_templates/design.md b/planning/_templates/design.md new file mode 100644 index 0000000..d63e22d --- /dev/null +++ b/planning/_templates/design.md @@ -0,0 +1,49 @@ +--- +summary: One line — shown in the generated index. Written at creation; finalize at ship to state the realized result. +--- + +# Design: One-line capitalized title + +## Summary + +One paragraph. What changes, at the level a reader needs to decide if this +spec is worth reading in full. + +## Motivation + +Why now. What is broken or missing. Concrete observations / numbers, not +abstract complaints. Link to memory entries or earlier specs when relevant. + +## Non-goals + +What is deliberately out of scope and (when nontrivial) why. Each item is +a sentence; one line each. + +## Design + +### 1. + +What changes, in enough detail that a reader who has not seen the codebase +can follow. Code samples / diagrams welcome. + +### 2. + +... + +## Operations + +Out-of-repo steps (DNS, infra, external account changes). Omit if none. + +## Out of scope + +Already covered above under Non-goals if appropriate. Repeat-list of +explicitly-excluded follow-ups belongs here when the list is long. + +## Testing + +How we know it landed correctly. New pytest? Smoke check on live URL? +Lint pass? Be specific. + +## Risk + +What could go wrong, ranked by likelihood × impact. Mitigations. diff --git a/planning/_templates/plan.md b/planning/_templates/plan.md new file mode 100644 index 0000000..132d720 --- /dev/null +++ b/planning/_templates/plan.md @@ -0,0 +1,46 @@ +# — implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use +> superpowers:subagent-driven-development (recommended) or +> superpowers:executing-plans to implement this plan task-by-task. Steps +> use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** One sentence — what shipping this plan achieves. No design +rationale; link to the spec for that. + +**Spec:** [`design.md`](./design.md) + +**Branch:** `feat/my-change` (or `fix/`, `chore/`, etc.) + +**Commit strategy:** Per-task commits / single commit / squash on merge. +Whichever fits. + +--- + +### Task 1: + +**Files:** +- Modify: `path/to/file.py` +- Create: `path/to/new.py` + +One sentence on what this task accomplishes. No deeper reasoning — that's +in the spec. + +- [ ] **Step 1: ** + + Run / edit / verify command. Expected output. + +- [ ] **Step 2: ** + + ... + +- [ ] **Step 3: Commit** + + ```bash + git add path/to/file.py + git commit -m ": " + ``` + +--- + +### Task 2: ... diff --git a/planning/_templates/release.md b/planning/_templates/release.md new file mode 100644 index 0000000..5081187 --- /dev/null +++ b/planning/_templates/release.md @@ -0,0 +1,38 @@ +# — + + + + + +## Feature + +- **.** What it adds and how to use it. + +## Fix + +- **.** What was broken, now fixed (reference the issue/regression). + +## Internal refactors + +- **.** What changed under the hood, stated as no behavior change. + +## Packaging + +- Metadata / build / dependency changes visible to installers. + +## Why + +Context a reader needs for the headline change. Omit for small releases. + +## Downstream + +What dependents must do — e.g. bump their version floor — or "No action +needed" when there is no API change. Omit if the project has no downstreams. + +## Internals + +- Coverage / tooling notes. diff --git a/planning/changes/.gitkeep b/planning/changes/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/planning/decisions/.gitkeep b/planning/decisions/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/planning/deferred.md b/planning/deferred.md new file mode 100644 index 0000000..b89f0e7 --- /dev/null +++ b/planning/deferred.md @@ -0,0 +1,4 @@ +# Deferred + +Real-but-unscheduled items. Each entry names a concrete revisit trigger — the +signal that should pull it back onto the board. Empty until something lands here. diff --git a/planning/index.py b/planning/index.py new file mode 100644 index 0000000..a1632e1 --- /dev/null +++ b/planning/index.py @@ -0,0 +1,193 @@ +# ruff: noqa: INP001, D212 # planning/ is not a Python package; D212/D213 conflict differs from faststream-outbox +""" +Generate the planning index from frontmatter. + +Run via ``just index``. Globs ``planning/changes/*/`` (each bundle's +``design.md``, falling back to ``change.md``) and ``planning/decisions/*.md``, +reads their frontmatter, and prints a Markdown listing to stdout — changes +then decisions, newest-first. Never writes a file: +the listing is a query over the files, not a committed artifact. + +``date`` and ``slug`` are derived from the directory / file name, not +frontmatter — the name is the single source of truth for both. +""" + +import pathlib +import re +import sys + + +CHANGES_DIR = pathlib.Path(__file__).parent / "changes" +DECISIONS_DIR = pathlib.Path(__file__).parent / "decisions" +VALID_DECISION_STATUS = {"accepted", "superseded"} +BUNDLE_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})\.\d{2}-(?P.+)$") +DECISION_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})-(?P.+)$") +ALLOWED_BUNDLE_FILES = {"design.md", "plan.md", "change.md"} +SPEC_REQUIRED = ("summary",) +DECISION_REQUIRED = ("status", "summary") + + +def parse_frontmatter(text: str) -> dict[str, str]: + """Parse a single-line-scalar YAML frontmatter block into a dict.""" + lines = text.splitlines() + if not lines or lines[0].strip() != "---": + return {} + fields: dict[str, str] = {} + for line in lines[1:]: + if line.strip() == "---": + break + if line[:1] in (" ", "\t"): + continue + key, sep, value = line.partition(": ") + if not sep: + continue + cleaned = value.strip().strip('"').strip("'") + fields[key.strip()] = "" if cleaned == "null" else cleaned + return fields + + +def _named(fields: dict[str, str], name: str, pattern: re.Pattern[str]) -> dict[str, str]: + """Inject ``date``/``slug`` derived from a dir/file name into ``fields``.""" + match = pattern.match(name) + if match: + fields["date"] = match.group("date") + fields["slug"] = match.group("slug") + return fields + + +def load_bundles() -> list[dict[str, str]]: + """Read each bundle's summary; derive date/slug from the directory name.""" + bundles: list[dict[str, str]] = [] + if not CHANGES_DIR.is_dir(): + return bundles + for bundle in sorted(CHANGES_DIR.iterdir()): + if not bundle.is_dir(): + continue + spec = bundle / "design.md" + if not spec.exists(): + spec = bundle / "change.md" + if not spec.exists(): + continue + fields = _named(parse_frontmatter(spec.read_text(encoding="utf-8")), bundle.name, BUNDLE_RE) + fields["path"] = f"changes/{bundle.name}/{spec.name}" + fields["name"] = bundle.name + bundles.append(fields) + return bundles + + +def load_decisions() -> list[dict[str, str]]: + """Read each decision's frontmatter; derive date/slug from the file name.""" + decisions: list[dict[str, str]] = [] + if not DECISIONS_DIR.is_dir(): + return decisions + for path in sorted(DECISIONS_DIR.glob("*.md")): + if path.name == "README.md" or path.name.startswith("_"): + continue + fields = _named(parse_frontmatter(path.read_text(encoding="utf-8")), path.stem, DECISION_RE) + fields["path"] = f"decisions/{path.name}" + fields["name"] = path.stem + decisions.append(fields) + return decisions + + +def format_row(bundle: dict[str, str]) -> str: + """Render one bundle as a Markdown list item.""" + slug = bundle.get("slug", "?") + path = bundle.get("path", "") + date = bundle.get("date", "") + summary = bundle.get("summary") or "(no summary)" + line = f"- **[{slug}]({path})** ({date}) — {summary}" + if bundle.get("supersedes"): + line += f" _(supersedes {bundle['supersedes']})_" + if bundle.get("superseded_by"): + line += f" _(superseded by {bundle['superseded_by']})_" + return line + + +def render(bundles: list[dict[str, str]], decisions: list[dict[str, str]]) -> str: + """Render the full Markdown listing: changes then decisions, newest-first.""" + out = ["# Planning index", "", "_Generated by `just index` — do not edit._", "", "## Changes", ""] + change_rows = sorted(bundles, key=lambda b: b.get("name", ""), reverse=True) + out += [format_row(b) for b in change_rows] if change_rows else ["_None._"] + out += ["", "## Decisions", ""] + decision_rows = sorted(decisions, key=lambda d: d.get("name", ""), reverse=True) + out += [format_row(d) for d in decision_rows] if decision_rows else ["_None._"] + out.append("") + return "\n".join(out).rstrip() + "\n" + + +def _require(fields: dict[str, str], keys: tuple[str, ...], rel: str, violations: list[str]) -> None: + """Append a violation for each required key that is absent or empty.""" + violations.extend(f"{rel}: missing or empty frontmatter key '{key}'" for key in keys if not fields.get(key)) + + +def _check_spec_file(path: pathlib.Path, rel: str, violations: list[str]) -> None: + """Validate a design.md / change.md spec file (requires `summary`).""" + fields = parse_frontmatter(path.read_text(encoding="utf-8")) + _require(fields, SPEC_REQUIRED, rel, violations) + + +def _check_bundle(bundle: pathlib.Path, violations: list[str]) -> None: + """Validate one change bundle directory.""" + rel = f"changes/{bundle.name}" + if BUNDLE_RE.match(bundle.name) is None: + violations.append(f"{rel}: directory name is not 'YYYY-MM-DD.NN-slug'") + violations.extend( + f"{rel}/{child.name}: unexpected file in bundle (allowed: {', '.join(sorted(ALLOWED_BUNDLE_FILES))})" + for child in sorted(bundle.iterdir()) + if child.name not in ALLOWED_BUNDLE_FILES + ) + design = bundle / "design.md" + change = bundle / "change.md" + if not design.exists() and not change.exists(): + violations.append(f"{rel}: bundle has neither design.md nor change.md") + for spec_file in (design, change): + if spec_file.exists(): + _check_spec_file(spec_file, f"{rel}/{spec_file.name}", violations) + # plan.md carries no frontmatter — its identity comes from the bundle dir. + + +def _check_decision(path: pathlib.Path, violations: list[str]) -> None: + """Validate one decision file (requires `status` + `summary`).""" + rel = f"decisions/{path.name}" + if DECISION_RE.match(path.stem) is None: + violations.append(f"{rel}: file name is not 'YYYY-MM-DD-slug.md'") + fields = parse_frontmatter(path.read_text(encoding="utf-8")) + _require(fields, DECISION_REQUIRED, rel, violations) + status = fields.get("status", "") + if status and status not in VALID_DECISION_STATUS: + violations.append(f"{rel}: invalid status '{status}' (allowed: {', '.join(sorted(VALID_DECISION_STATUS))})") + + +def check() -> list[str]: + """Validate every bundle and decision; return the list of violation strings.""" + violations: list[str] = [] + if CHANGES_DIR.is_dir(): + for bundle in sorted(CHANGES_DIR.iterdir()): + if bundle.is_dir(): + _check_bundle(bundle, violations) + if DECISIONS_DIR.is_dir(): + for path in sorted(DECISIONS_DIR.glob("*.md")): + if path.name == "README.md" or path.name.startswith("_"): + continue + _check_decision(path, violations) + return violations + + +def main() -> int: + """Print the listing to stdout, or validate bundles with --check.""" + if "--check" in sys.argv[1:]: + violations = check() + if violations: + sys.stderr.write(f"planning: {len(violations)} violation(s)\n") + for violation in violations: + sys.stderr.write(f" - {violation}\n") + return 1 + sys.stdout.write("planning: OK\n") + return 0 + sys.stdout.write(render(load_bundles(), load_decisions())) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/planning/releases/.gitkeep b/planning/releases/.gitkeep new file mode 100644 index 0000000..e69de29