diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 3a965e5..f1720f7 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,237 +1,69 @@ -# Knowledge flush — 13 insight(s) ingested (21 claimed, 2 dropped, 6 released) +# Knowledge flush — 12 insight(s) -Cross-Check: 1× independent adversarial `claude` CLI headless pass over the 5 new pages — it refuted the changed-files gate page's "prettier exits 0 on an empty match set" claim; re-measured against Prettier 3.7.4, confirmed the reviewer was right (unmatched operand exits **2**), and rewrote the page, report rows 3/5 and `log.md`. Other 5 claim groups verdicted sound. Limits: the reviewer's sandbox blocked repo reads, so source-quote-supports-directive and self-contradiction dimensions went unaudited (details in `## Cross-Check`). - -Queue drained under run id `20260827-125731-38371` (this session is the detached -`hooks/auto-flush.sh` run; its step-0 acquire resolved re-entrantly to -`already-owned`, not to a competing holder). 21 rows were claimed; 13 are -ingested below, 2 are retired as out-of-layer, and 6 are released back to -`pending` for a later flush because each needs its own page rather than a row, -and rushing six more pages in one pass would have lowered the bar on all of them. +Run `20260906-013758-50852` (headless auto-flush), claimed 12 of 41 pending rows via `queue-claim.js claim --max 12`. Outcome: 5 new pages, 3 merges into existing pages, 3 folds pushed onto open PR #181, 1 pending-duplicate drop (PR #185). Lint on this branch: `wiki-structure-checks` findings 0 (281 pages), `wiki-lint-prohibitions` violations 0, `wiki-lint-model-era` 22 coupled / 21 candidates (the new model-era page carries `verified_model`). ## Verified best-practice -Every external claim below was live-fetched this session and quoted in the page's -`Sources` block. Field evidence carries the repo, date, and the measured numbers. - -| # | Claim | Sources checked | How verified | Confidence | -|---|-------|-----------------|--------------|------------| -| 1 | `now()` is `transaction_timestamp()` (fixed at transaction start) while `clock_timestamp()` "changes even within a single SQL statement"; `RETURNING` yields computed defaults "without needing a separate database query" | postgresql.org `functions-datetime`, `dml-returning`, `transaction-iso` | Fetched; both key sentences quoted verbatim into the page | verified | -| 2 | A boundary recomputed in a follow-up step is a *second, later* `now`, widening a `<= boundary` set | Field: `rtb-unified` `packages/orpc/src/routers/batch.ts` — codifies "one `now` per decision" and passes `now` into the boundary helper; its result type omits the boundary, which is the shape that invites recomputation | Read the invariant and the signature in the cited file | field-tested | -| 3 | **[CORRECTED BY CROSS-CHECK]** The vacuous-pass shapes for `prettier --check` are: no operands (rc **0**), all operands ignore-filtered (rc **0**), and unsupported extensions with `--ignore-unknown` (rc **0**). A pattern/operand matching nothing exits **2** — it prints the success sentence *and* an unmatched-pattern error | prettier.io CLI + ignore docs; local measurement, Prettier 3.7.4 | The first draft generalised "empty match set ⇒ exit 0" from a field log where both messages appeared together. The independent reviewer flagged it; I then ran all seven cases against a real binary and rewrote the page around the measured table | verified (re-measured) | -| 4 | zsh does not word-split unquoted parameter expansions by default, so `cmd $FILES` arrives as **one** operand | zsh FAQ ch. 3 (`SH_WORD_SPLIT`) | Fetched; quoted ("By default, zsh does not have that behaviour: the variable remains intact") | verified | -| 5 | The zsh word-split operand exits **2**, but its log still carries the success sentence — so the log misleads even though the exit code does not | Field 2026-08-24 (`rtb-unified`, zsh) + local measurement 2026-08-27 | Field log showed both messages together; the local run reproduced it as `rc=2`. The page now says explicitly that this row fails loudly *unless* `--no-error-on-unmatched-pattern` is set. Probe placement re-confirmed: `.claude/tmp/` is `.gitignore`d, so a probe there passes at rc 0 | verified (re-measured) | -| 6 | TypeScript applies excess-property/contextual typing to fresh object literals, so a value of a type can be constructed with the type's name absent from the text | typescriptlang.org handbook, *Object Types* | Fetched; confirmed the check follows from the contextual type, not from a written annotation | verified | -| 7 | `tsc`'s program is `files` ∪ `include` ∪ transitive imports; `exclude` "only changes which files are included as a result of the `include` setting" and does not stop an imported file entering the program | typescriptlang.org TSConfig `#include`, `#exclude` | Fetched; the `exclude` sentence quoted (it sharpens the rule to "in the program", not "in `include`") | verified | -| 8 | Consequence of 6+7 measured | Field 2026-08-24/25 (`rtb-unified`): `grep "DealViewer"` reported 3 construction sites, actual 8 — the missed set included production wiring `routers/deal.ts:38`; `ContractScopeActor` 7→~22. Separately, `packages/orpc/tsconfig.json` `include: ["src/**/*"]` produced 3 production + 13 api-test errors and **zero** for `__tests__/routers/deal.test.ts`, whose 6 sites appeared only as 6 failing tests | Counts recorded from the cited runs | verified | -| 9 | cgroup v2: `memory.peak` is max usage since creation/reset; at `memory.max` "the OOM killer is invoked in the cgroup"; in `memory.events`, `max` counts times usage "was about to go over the max boundary" — **distinct** from `oom_kill` | docs.kernel.org cgroup-v2 admin guide | Fetched; all four quoted. This corrected the candidate, which had read a non-zero `max` as a kill; the page now states the distinction explicitly | verified | -| 10 | An `exec`'d process joins the container's cgroup and is invisible to the application's own semaphore | kubernetes.io `manage-resources-containers`, `assign-memory-resource`, `kubectl exec` reference + field 2026-08-26 (review-bot pod, `limits.memory: 3Gi`): `memory.current` 2.54 GiB, `memory.peak` 3.0 GiB (at the limit), `memory.events: max 5`, while `maxConcurrentAgents: 20` reported free slots | Docs fetched; pod numbers from the cited measurement | verified | -| 11 | Basename-keyed mutation backups collide across directories and restore cross-writes; an untracked file's `git diff` is empty whether restored or destroyed | Field 2026-08-21 (`rtb-unified`, NEWRTB-2936): restore wrote `schemas/deal.ts` into `routers/deal.ts` → `Cannot find module './common.js'`, `grep -c dealRouter` = 0; **both files were 154 lines**, so a line-count check passed; after re-keying, M9/M10 flipped SURVIVED→KILLED. Plus stryker mutant-states / pitest for the verdict vocabulary | Reproduced end to end in the cited run | field-tested | -| 12 | A negative assertion is vacuous when the fixture never supplies the triggering input | Field 2026-08-25 (`rtb-unified`): with `staleQueuedJobIds: []` the code early-returned; the widening the assertion claimed to catch survived 116/116 green | Mutation applied and observed | field-tested | -| 13 | A body-level (non-inline) review finding cites no file, so rebutting from an assumed file rejects real defects | Field 2026-08-19 (PR #327 r16): quote matched `report.py:393/416/425`, not the assumed `fill_plan.py:307` — sibling modules, one already fixed | Grep resolved the quote to the real site | field-tested | -| 14 | Unifying two duplicate allowlists defaults to the union and silently widens each side | Field 2026-08-25 (`rtb-unified` PR #965): folding `DISPLAYABLE_ERROR_CODES` into `USER_FACING_ERROR_CODES` would have added `UNAUTHORIZED` + `VALIDATION_ERROR`, exposing raw server messages as inline UI errors; caught only by computing the difference first | Difference computed before the merge | field-tested | - -Not upgraded: nothing was marked `verified` on field evidence alone. Two pages -carry `confidence: field-tested` (`mutation-harness-file-custody`, -plus the pre-existing `evaluating-review-feedback`), and no candidate was -recorded as `verified` without a fetched primary source. +1. **SwiftPM: test an executable target directly** (`8cfed7ce6d1fb5dc`) — claim: since tools-version 5.5 a test target may depend on an `.executableTarget` and `@testable import` it. Sources fetched and quote-checked by curl: swift-package-manager CHANGELOG (`#3316` "Test targets can now link against executable targets as if they were libraries … tools version of `5.5` or newer"; `#4119` `--disable-testable-imports`), SE-0294 (Implemented Swift 5.4), TSPL AccessControl.md (`@testable` needs "compiled with testing enabled"), SPMBuildCore `BuildParameters+Testing.swift` (`explicitlyEnabledTestability ?? (configuration == .debug)`), swift-package-manager#6367 (works on macOS/Linux; Windows `duplicate symbol: main`), Swift Forums 52351. Field: desk-bat 49/49. **verified**. +2. **macOS off-screen capture without Screen Recording** (`49e0bd8194f59531`) — claim: render the app's own content (`SKView.texture(from:)`, `NSView.cacheDisplay`) and assemble GIFs with ImageIO instead of `screencapture` when the TCC grant is unavailable. Apple doc JSON endpoints quote-checked: `texture(from:)` ("does not need to appear in the view's presented scene"), `SKRenderer`, `cacheDisplay(in:to:)`, `kCGImagePropertyGIFDelayTime` ("clamped to a minimum of 100 milliseconds" — this corrected the candidate's 15 fps cadence to 10 fps), macOS 15.1 release notes ("deprecated content capture technologies now have enhanced user awareness policies"), WWDC19 701 transcript ("preapprove apps to record the entire screen or the contents of windows other than their own"), Apple Platform Security guide (Screen recording is TCC-gated). The literal `could not create image` text and the relaunch-after-grant step are field evidence only and labelled so. **verified**. +3. **Testcontainers reaper on Docker Desktop macOS** (`2bea722bd3446ce7`) — candidate said "disable Ryuk". Research (testcontainers-python README, java/go configuration docs, Rancher Desktop + Colima guides, docs.docker.com Advanced settings, testcontainers-java#8170/#7678, testcontainers-go#399, all quote-checked) shows the documented fixes are the Desktop "Allow the default Docker socket to be used" setting or `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock` with Ryuk kept on; disabling Ryuk is the last resort and "will prevent testcontainers from automatically cleaning up resources". Page orders the rows accordingly; the candidate's 0 tests → 27/27 measurement is kept as field evidence. **verified**. +4. **Ciphertext orphaned by a regenerated key** (`83a8c321f97b084c`) — sources quote-checked: OWASP Cryptographic Storage Cheat Sheet (key-id-with-ciphertext, retain old keys, rotation procedures in place beforehand), Node crypto docs (`decipher.final()` throws on failed authentication — a wrong key is indistinguishable from tampering), Rails ActiveRecord Encryption `previous:`, Tink 5-byte key-id prefix. OWASP Key Management sheet and NIST SP 800-57 were fetched but yielded no quotable rotation text and are not cited. Field: 124/124 unrecoverable. **verified**. +5. **Model-coupled guidance aging detector** (`75447f854be33d2d`) — sources: Anthropic prompting best practices (guidance ages per generation: "Tools that undertriggered in previous models are likely to trigger appropriately now"), dev-loop `scripts/wiki-lint-model-era.js` (merged PR #178, whose body records 3/271 phrase hits vs ~27/271 subject hits). Reproduced on this checkout: `pages: 276, model-coupled: 21, candidates: 21`, exit 3; whole-file grep 32 vs scoped 21 (supports the prose-only scoping directive). **verified**. +6. **CORS preflight silently drops an injected probe's JSON POST** (`bac1bbff54ea37c6`) — mechanism already sourced on the existing page (MDN CORS, Fetch spec: `application/json` is not safelisted → `OPTIONS` preflight); added the diagnostic rows and the field reproduction (0 → 6 reports after handling `OPTIONS`). Page stays **verified**. +7. **Queued candidate already landed in the store** (`694be184a03a9676`) — evidence checked in `~/.dev-loop/queue/.processed.jsonl` (row `f1ba9bf617fbd101` status `dropped-already-merged`), `code-graph-as-orientation-layer.md` line 102 on `main`, PR #184 merged 2026-09-04. Generalised as a control-signal-vs-primary-artifact row. **field-tested** row on a verified page. +8. **Literal re-assertion of a layout constant** (`ea0d83f672d08fa2`) — the never-fails pattern is an instance of tests-that-cannot-fail's thesis; field evidence (LinklyTabBar 96 vs 80 pt, fix 4e584cc) added as a row + Sources bullet. **field-tested** row on a verified page. +9. **Grep both ends of a threaded value** (`9a3ec3b23171ebdd`) and 10. **enumerate contract-test contrast pairs** (`204952e682bc7e76`) — plan-authoring lenses; folded as author-side rows into #181's checkable-claims page whose WCAG source already covers the ratio arithmetic. **field-tested** rows. +11. **CJK label + pill wraps after padding change** (`418bcb06dbb3562e`) — CSS Text 3 ("line breaking conventions allow the line to break anywhere except between certain character combinations"), MDN word-break `keep-all` contrast, MDN flex-shrink, quote-checked. **verified** rows, folded into #181's responsive-layout. +12. **Prisma `CREATE INDEX CONCURRENTLY` in a multi-statement migration** (`11c6ad7b2166e611`) — not re-researched: open PR #185 already carries the same directive and the same 2026-08-31 field evidence in `online-schema-changes.md`. Dropped. ## Existing-layer check -Method: routed via `INDEX.md` → domain `index.md`; then built a full id+title -index of all 265 pre-existing pages and probed it with concept greps -(`clock_timestamp|clock skew`, `changed[- ]files|--ignore-unknown`, `tsconfig`, -`contextual typ|excess property`, `set difference|allowlist`, `cgroup`, -`basename|backup.*restore`, `2>&1`, `delta|baseline`) before deciding new vs merge. - -Pages read: testing-quality-source-text-wiring-assertions, testing-quality-tests-that-cannot-fail, backend-common-change-impact-call-site-enumeration, backend-common-change-impact-widening-a-closed-value-table, qa-process-evaluating-review-feedback, infrastructure-containers-host-cgroup-visibility, testing-quality-behavior-not-implementation +Pages read: infrastructure-agent-orchestration-control-signals-vs-primary-artifacts, backend-common-api-design-cors-and-preflight, qa-environments-browser-console-capture-gaps, debugging-methodology-probe-path-vs-operation-path, frontend-design-responsive-layout, backend-common-change-impact-call-site-enumeration, testing-quality-tests-that-cannot-fail, testing-quality-value-preserving-refactor-assertions, security-secrets-secrets-in-code, databases-schema-design-online-schema-changes, qa-document-verification-editing-a-gated-document, testing-quality-guard-shape-vs-consequence, qa-environments-test-environment-parity, mobile-security-sensitive-data-on-device -Findings: +Also read on open-PR branches only (not on this checkout's `main`): `checkable-claims-in-an-adopted-plan` (#181), `element-crop-screenshots` (#182), `online-schema-changes` as changed by #185. All ten domain `index.md` files and `INDEX.md` were read for routing. -- **Zero coverage** (→ new pages): changed-files-only gates, tsconfig/contextual - typing, allowlist set-difference, app-clock-vs-DB-timestamp, exec-into-a-running-container. - The concept greps returned no hits for these; the clock hits were incidental - (offline sync, token handling) and none compared an app clock to a DB column. -- **Already covered — one candidate all but retired.** The comment-stripping - insight is `source-text-wiring-assertions` step 2 verbatim ("Make the - assertion's subject the file with comments removed"), and its false-RED and - negative/count false-GREEN shapes are already edge rows. Only the *empty-slice* - consequence was new, so that alone was merged. -- **Line-cap conflict handled without breaking the invariant.** - `source-text-wiring-assertions` sits at exactly **120** body lines (the - documented cap). Rather than add a row and violate maintenance invariant 5, the - new nuance and the new field evidence were merged **in place** into an existing - edge row and an existing source bullet. Body count re-measured after editing: - still 120. -- **No conflicts found.** Nothing ingested contradicts an existing directive. - The one correction made was to a *candidate*, not to the wiki (item 9: the - `memory.events` `max` counter is approaches-to-limit, not kills). -- **Related links added both ways**: `tests-that-cannot-fail` ↔ - `mutation-harness-file-custody`; `widening-a-closed-value-table` ↔ - `compiler-as-call-site-inventory` (+ `errors-diagnostics-from-a-shared-code-path`); - `host-cgroup-visibility` → `exec-added-processes-and-the-memory-budget` - (from its existing self-monitoring row). -- **Indexes/log updated**: 4 domain indexes (+5 "load when" rows), `log.md` - appended. Root `INDEX.md` unchanged — no new domain. - -Gates run (the exact CI commands from `.github/workflows/test.yml`): -`node scripts/wiki-structure-checks.js wiki` → **pages: 270, indexes: 13, -findings: 0**; `node scripts/wiki-lint-prohibitions.js wiki` → **directives 72, -compliant 72, violations 0** (the 1 `info` is pre-existing in -`config/keys-ahead-of-their-consumer.md`, untouched); `bash scripts/check-versions.sh` -→ `ok: dev-loop 1.11.2`. The `bats tests/` job was **not** run — bats is not -installed on this machine, and this change touches only wiki markdown (no -scripts or hooks), so that suite's subject is unchanged. +Overlaps and decisions: +- CORS: `cors-and-preflight` already states the `application/json` → preflight mechanism; merged the diagnostic angle (probe silence) as an edge-case row + Instead-of row + field source, and linked `debugging-methodology-probe-path-vs-operation-path` and `qa-environments-browser-console-capture-gaps` from its `related:`. Back-links onto those two pages skipped because #180 and #182 rewrite their frontmatter. +- Queue lag: `control-signals-vs-primary-artifacts` item 1 ("confirm the claim against the primary artifact") is the exact frame; added a Confirm-with row, an edge-case row and an Instead-of row. Body-only — #179 rewrites this page's `related:`. +- Literal constant: `tests-that-cannot-fail` never-fails table is the home; `value-preserving-refactor-assertions` covers the adjacent (literal moved to config) case and was read to confirm no overlap. Body-only — #179/#180 rewrite the frontmatter. +- SwiftPM: no page mentions SwiftPM/`@testable`/executable targets (grep). New page under testing/strategy; related to `test-level-choice` (subprocess vs import level). +- Off-screen capture: no page mentions TCC/screencapture/SpriteKit (grep; `sensitive-data-on-device` mentions app-switcher screenshots only). New page under qa/environments beside the console-capture and element-crop pages; back-link added on `sensitive-data-on-device`. +- Testcontainers: no page mentions testcontainers/ryuk (grep); `test-data-and-isolation` covers fixtures/isolation, not the container runtime. New page under testing/data (no new category — re-checked qa/environments `test-environment-parity`, which is about staging parity, and linked it both ways). +- Key mismatch: security/secrets has only `secrets-in-code` (leak/storage); no encryption-at-rest page. New page; back-link added on `secrets-in-code`; `related:` to hypothesis-testing and probe-path (back-links skipped: #179/#180 touch those frontmatters). +- Model-era detector: no page on `main` covers it (grep for model-era/model-coupled: 0). New page under qa/document-verification; back-links added on `editing-a-gated-document` and `guard-shape-vs-consequence`. +- Conflicts flagged: none. No existing directive is contradicted; the testcontainers page demotes the candidate's own directive (disable Ryuk) to the last row, recorded in `log.md`. ## Open-PR check -`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"` -returned **no open PRs**, and a second unfiltered `gh pr list --state open` -returned none either — the repository has zero open PRs at flush time. There -were therefore no in-flight sibling branches to diff against, and no -`git fetch origin ` / `git diff origin/main origin/ -- wiki/` -comparisons to run. - -Per-candidate verdict: **all 21 = `new`.** No `fold`, no `drop-as-pending-duplicate`. -(The 2 drops recorded below are out-of-layer drops, not pending-duplicate drops.) - -## Routing decision - -**New pages (5)** - -| Page | Domain/category | From | Why not an existing page | -|------|-----------------|------|--------------------------| -| `application-clock-vs-database-timestamps` | databases / transactions | `2b27d15d` + `bea92fdd` | No page compares an app clock to a DB column. `transactions` chosen over `schema-design` because the decisive content is transaction-time semantics (`now()` = transaction start ⇒ stamp order ≠ commit order) and the fix is a lock/isolation choice | -| `changed-files-only-gates` | infrastructure / ci-cd | `ff041061` + `4b9af3a0` | Zero grep hits. Both candidates are the same defect (a gate green with an empty subject) from two directions, so they became one page rather than two | -| `compiler-as-call-site-inventory` | backend / common / change-impact | `702dcf4e` + `94d55f2f` | `call-site-enumeration` is the sibling case (callers of a changed signature, Python positional-vs-keyword) and is at 80 body lines; the TS mechanism is *constructors of a type* with its own workflow, so per "one case per page" it is a separate page, cross-linked | -| `mutation-harness-file-custody` | testing / quality | `6a9de235` + `41fa1c87` | `harness-reverse-controls` covers scoring a harness; nothing covers the harness's custody of the tree. Both candidates are that one case (keying, and the read window) | -| `exec-added-processes-and-the-memory-budget` | infrastructure / containers | `7b9e8788` | `host-cgroup-visibility` is cross-pod read mechanics and explicitly routes self-monitoring elsewhere; `resource-limits-and-probes` is manifest authoring. This is a runtime preflight before adding load | - -No new category was created — all five landed in existing categories. - -**Merged into existing pages (5 candidates)** - -| Candidate | Merged into | Shape | -|-----------|-------------|-------| -| `91ef5d53` | `testing-quality-tests-that-cannot-fail` | +1 never-fails row, +1 Instead-of row, +1 source | -| `f189f423` | `testing-quality-source-text-wiring-assertions` | In-place extension of 1 edge row + 1 source bullet (page at the 120-line cap) | -| `bb6d8539` | `backend-common-change-impact-widening-a-closed-value-table` | +Do-this 6 & 7 (incl. a set-difference ruling table), +1 Instead-of row, +1 source | -| `60a817ee` | `qa-process-evaluating-review-feedback` | +2 edge rows, +1 Instead-of row, +1 source | -| `7b9e8788` | `infrastructure-containers-host-cgroup-visibility` | Cross-link from its self-monitoring row to the new page | +Open `knowledge/*` heads listed with `gh pr list --search "head:knowledge/"`: #185 `knowledge/choiyounggi-20260906-003745`, #183 `…20260904-133717`, #182 `…20260903-214027`, #181 `…20260903-203836`, #180 `…20260903-184706`, #179 `…20260903-172728`. Each head was fetched and `git diff origin/main origin/ -- wiki/` inspected. -**Dropped — out of layer (2, retired)** +| Candidate | Overlapping open head | Verdict | +|-----------|-----------------------|---------| +| `11c6ad7b2166e611` Prisma CONCURRENTLY | #185 rewrites `online-schema-changes.md` with the same two rows and the same 2026-08-31 field evidence | **drop** (pending duplicate) | +| `9a3ec3b23171ebdd` grep both ends | #181 adds `checkable-claims-in-an-adopted-plan.md` (plan claims to check) | **fold** — rows pushed to #181 (commit on that branch + PR comment) | +| `204952e682bc7e76` contrast pairs | #181 same page (WCAG recomputation item 1) | **fold** — row pushed to #181 | +| `418bcb06dbb3562e` CJK pill | #181 rewrites `responsive-layout.md` frontmatter + rows | **fold** — rows pushed to #181 to avoid a sibling conflict; #181 did not previously carry the insight | +| `694be184a03a9676` queue lag | #179 rewrites `control-signals-vs-primary-artifacts` `related:` only | **new** (body-only merge here) | +| `ea0d83f672d08fa2` literal constant | #179/#180 rewrite `tests-that-cannot-fail` `related:` only | **new** (body-only merge here) | +| `bac1bbff54ea37c6` CORS probe | none touch `cors-and-preflight.md` | **new** | +| `8cfed7ce6d1fb5dc`, `49e0bd8194f59531`, `2bea722bd3446ce7`, `83a8c321f97b084c`, `75447f854be33d2d` | none (grep of every open head's wiki diff for swiftpm/testable, TCC/screencapture/SpriteKit, ryuk/testcontainers, AES/decrypt/ciphertext, model-era/verified_model: 0 hits) | **new** | -- `094dedf3` — a Figma MCP `inspect_node` → `get_dev_ready` children-fetch - workaround. The server is a private, org-internal MCP plugin; the behavior is - not publicly verifiable and the directive does not transfer to any other reader. -- `e165a365` — an `/rtb:review` remote-fallback runbook naming - `~/.claude/tools/rtb-remote-review.sh` and an internal pod. The transferable - kernel ("a two-provider review gate degraded to one provider is not a passed - gate") is already the subject of `qa-process-llm-review-pipelines`; what remains - is machine-specific paths. +Index files (`wiki/*/index.md`, `INDEX.md`) are also touched by every open PR; those are additive-row conflicts for the owner to resolve at merge, as in the previous flushes. -**Released back to `pending` (6)** — each needs its own page, not a row, and is -better served by a dedicated pass than by being appended here: -`81dc1f98` (naming the carrier field/type when a plan says "wire A to B"), -`b9ae304a` (`VAR="$(cmd 2>&1)"` mixing stderr into a value used as a path), -`fdd0b3c6` (monitor markers anchored at line start; delta rather than absolute -state; first cycle records a baseline), -`c2adb2be` (positional-order assertions on rendered SQL predicates), -`815e8cb9` (grep only *active* `DATABASE_URL` assignments, and confirm which -dotenv file the tool loads, before a destructive DB command), -`f1146adb` (CI ticket-key extraction scoped by changed-file intersection rather -than by mention). - -## Decision Log - -**Intent.** Drain the harvested `★ Insight` queue into reviewable wiki knowledge -without lowering the wiki's evidence bar. The queue held 21 rows accumulated over -several days; the goal was correct routing and real verification, not a high -ingest count. - -**Alternatives considered and rejected.** - -- *Ingest all 21 in this pass.* Rejected: six of them each need their own page, - and writing six more pages in one pass would have produced thin, weakly-sourced - entries. They are released to `pending`, not dropped, so the next flush takes - them with a full budget. -- *Append the two TypeScript candidates to `call-site-enumeration`.* Rejected: - that page is the sibling case (callers of a changed signature, Python - positional-vs-keyword). AGENTS.md requires one case per page, so the - constructor-enumeration case became its own page, cross-linked both ways. -- *Add a row to `source-text-wiring-assertions` for the empty-slice nuance.* - Rejected: that page is at exactly the documented 120-line body cap, so adding a - line would violate maintenance invariant 5. The nuance was merged **in place** - into an existing edge row instead; body re-measured at 120. -- *Drop the comment-stripping candidate entirely as a duplicate.* Rejected: its - directive is already the page's step 2, but the empty-slice consequence - (vacuous **green**, not the documented noisy red) was genuinely absent. -- *Claim a cross-check exemption because this PR cannot merge itself.* Rejected — - see below; the check found a real error, which is the argument against exempting. -- *Push to `origin`* as the skill's snippet does. Not available: this contributor - has no write access to `choiyounggi/dev-loop` (403). Used the pre-existing - `fork` remote, which is how every prior knowledge branch here was published. -- *Branch name from `git config user.name`.* The skill's ASCII sanitisation of a - Korean name yields an empty string → `anon`, defeating the attribution the - branch name exists for. Used the gh login, matching existing branch names. - -**Where reviewers should look hardest.** - -1. `infrastructure/ci-cd/changed-files-only-gates.md` — rewritten after the - cross-check. The measured table is the load-bearing part; please sanity-check - it against your own Prettier version, since the exit codes are version-visible - behaviour rather than a documented contract. -2. `databases/transactions/application-clock-vs-database-timestamps.md` step 5–6 — - the claim that timestamp order is not commit order, and that the remedy is a - lock/isolation level rather than finer clock resolution. `[추정]` on the MySQL - `NOW()`/`SYSDATE()` row: taken from general MySQL semantics, not fetched this - session like the PostgreSQL pages were. -3. `widening-a-closed-value-table.md` Do-this 6–7 — this inserts a security-shaped - concern (allowlist widening) into a page whose original subject was value - tables. If that reads as two cases, it should be split. -4. The 2 dropped candidates — if you consider private-tooling runbooks in scope - for this wiki, they should be restored rather than retired. - -## Cross-Check - -Independent adversarial pass via `claude` CLI headless (separate process, no -shared context), prompted to refute rather than confirm, over the five new pages' -technical claims. - -**It found a real error, and the page was rewritten because of it.** The reviewer -challenged the claim that `prettier --check` exits 0 on an empty match set, -arguing an unmatched pattern errors by default and that exit-0 belongs to the -ignore-filtered case. I resolved it by measurement rather than by argument — -running all seven cases against Prettier 3.7.4 — and the reviewer was right: -an unmatched operand exits **2** (while still printing the success sentence), -whereas the genuine silent vacuous passes are no-operands, all-ignore-filtered, -and `--ignore-unknown`-with-unsupported-extensions. The page, this report's -rows 3 and 5, and the `log.md` entry were all corrected. - -Verdicts on the other five claim groups: **sound** (PostgreSQL clock semantics — -noted as if anything *understated*; zsh word-splitting; TS contextual typing; -`tsc` program membership incl. `exclude`-does-not-stop-imports; cgroup v2 -`max` vs `oom_kill` and `kubectl exec` cgroup placement). - -Stated limits of the check: the reviewer's sandbox denied it read access to -`~/.dev-loop/repo/wiki`, so it adjudicated the six claims as quoted in its prompt -and could **not** audit (b) whether each `Sources` quote supports the directive it -is cited for, or (c) whether any page contradicts its own edge-case rows. Those -two dimensions remain unreviewed by an independent party and are the residual -risk in this PR. A first attempt also returned only the session's Stop-hook -output rather than a verdict; that run was discarded rather than read as -"no findings". - -## Review notes +## Routing decision -- PR-only, as required: no merge, no push to `main`. -- Commit is under the contributor's own ambient git identity - (`최영기 `, gh `dch0202-rsquare`); no assistant identity - and no `Co-Authored-By` trailer. The branch uses the gh login because - sanitizing the Korean `user.name` to ASCII yields an empty string, which the - skill's snippet would have turned into `anon` — that would have defeated the - attribution the branch name exists for. -- Scope purity: only `wiki/**`, four domain indexes, `log.md`, and this report. - Two untracked leftovers from earlier flushes - (`.dev-loop/CROSSCHECK_FINDINGS.md`, `.dev-loop/fold-note-73.md`) were left - untouched and unstaged. +| Candidate | Target | +|-----------|--------| +| `8cfed7ce6d1fb5dc` | testing/strategy — new `executable-target-tests-in-swiftpm` (level/structure decision: import vs subprocess) | +| `49e0bd8194f59531` | qa/environments — new `offscreen-render-capture-without-screen-recording` (evidence capture environment, beside console-capture and element-crop pages; platforms rejected because the directive is about producing QA/doc evidence, not OS portability) | +| `2bea722bd3446ce7` | testing/data — new `testcontainers-reaper-on-docker-desktop-macos` (test-infrastructure containers; no new `testing/environments` category — qa/environments `test-environment-parity` linked instead) | +| `83a8c321f97b084c` | security/secrets — new `ciphertext-orphaned-by-a-regenerated-key` (key lifecycle; debugging linked via `related:`) | +| `75447f854be33d2d` | qa/document-verification — new `model-coupled-guidance-aging-detector` (a gate over documents; `verified_model: claude-fable-5-1` set because the page is itself model-coupled) | +| `bac1bbff54ea37c6` | backend/common/api-design — merged into `cors-and-preflight` | +| `694be184a03a9676` | infrastructure/agent-orchestration — merged into `control-signals-vs-primary-artifacts` | +| `ea0d83f672d08fa2` | testing/quality — merged into `tests-that-cannot-fail` | +| `9a3ec3b23171ebdd`, `204952e682bc7e76` | infrastructure/agent-orchestration — folded into #181's `checkable-claims-in-an-adopted-plan` | +| `418bcb06dbb3562e` | frontend/design — folded into #181's `responsive-layout` | +| `11c6ad7b2166e611` | dropped (pending duplicate of #185) | + +No new category was added. `INDEX.md` route-here lines for qa, testing and security were extended by one clause each; domain `index.md` rows added for the five new pages and load-when lines extended for the three merged pages. `log.md` carries one entry per page plus the fold and drop entries. diff --git a/INDEX.md b/INDEX.md index 80fee3f..e594fc2 100644 --- a/INDEX.md +++ b/INDEX.md @@ -15,10 +15,10 @@ follow the cross-pointers in their index or take the next matching seeded domain | [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, call-site enumeration before a contract change, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, consuming LLM APIs (completion validation, context budgeting), authoring agent-facing artifacts (binding instruction text, agent tool-surface granularity/parity), MAPE-aligned point-prediction calibration, consuming external-API responses, externally-owned defaults, object-storage references, sync-vs-async integration choice, WebSocket/SSE connection lifecycle) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps, packaging data files with `importlib.resources`) | | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility, agent-facing tool surfaces (WebMCP) | | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting), per-environment/path-valued config, multi-agent orchestration (worker liveness signals, shared run state, tmux pane delivery, completion gates, worktree-isolated workers, autonomous ask-vs-rule decisions, session context/token budgeting, a pre-built code knowledge graph as a freshness-gated orientation layer for planning) | -| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, test-before-code ordering, cases/assertions, cross-layer effect scoping, test data, mock decisions, flaky tests (release-process quality → qa) | -| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, evidence for completion claims, acting on code-review feedback, adversarial review of high-risk diffs, exploratory testing (guarded-path coverage, override matrices), scope-purity gates, sourcing deliverable documents from generated artifacts, verifying the quantitative claims in a document before publishing it, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) | +| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, test-before-code ordering, cases/assertions, cross-layer effect scoping, test data, mock decisions, flaky tests, test-infrastructure containers (Testcontainers) failing on the dev host, testing a SwiftPM executable target (release-process quality → qa) | +| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, evidence for completion claims, acting on code-review feedback, adversarial review of high-risk diffs, exploratory testing (guarded-path coverage, override matrices), scope-purity gates, sourcing deliverable documents from generated artifacts, verifying the quantitative claims in a document before publishing it, automated verification of document deliverables (spec/RFC gates), an aging detector for model-coupled agent guidance, capturing an app's own screen content without Screen Recording permission (writing automated test code → testing) | | [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) | -| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling, in-session agent tool exposure (prompt-injection blast radius), the author identity a commit publishes to a public repository, host-compromise triage / incident response (verifying assumed security agents, identifying masquerading processes) (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | +| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene (including ciphertext orphaned by a regenerated encryption key), dependency trust, PII handling, in-session agent tool exposure (prompt-injection blast radius), the author identity a commit publishes to a public repository, host-compromise triage / incident response (verifying assumed security agents, identifying masquerading processes) (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | | [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning | | [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time | diff --git a/log.md b/log.md index e2ee9ca..38996e2 100644 --- a/log.md +++ b/log.md @@ -104,3 +104,13 @@ Append-only. Format: `## [YYYY-MM-DD] ` and require a 2xx carrying `Access-Control-Allow-Origin`, `-Methods`, `-Headers`; then answer `OPTIONS` with 204 plus those headers and put `Access-Control-Allow-Origin` on the `POST` response as well | ## Instead of | If you are about to | Do this instead | Why | |---------------------|-----------------|-----| | Set `Access-Control-Allow-Origin: *` on an endpoint that also sets `Access-Control-Allow-Credentials: true` | Echo back the validated request `Origin` as a single explicit value | The Fetch spec's CORS protocol forbids the wildcard on a credentialed response, and browsers enforce this by blocking the response even if the server sends it | +| Conclude "the page never loads the probe" because the collector logged zero requests | Send one `curl -i -X OPTIONS` to the collector and one deliberate `fetch` from the page's console | A collector that 404s the preflight and a probe that never ran produce the same empty log; the OPTIONS probe separates them ([debugging-methodology-probe-path-vs-operation-path]) | | Debug a CORS failure by relaxing the server to allow every origin | Read the exact console error (it names the missing/mismatched header) and add only that header for the specific origins that need it | A blanket allow-all reopens the endpoint to any site's browser-side JS, including credentialed requests if cookies are involved | ## Sources - https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS — simple vs preflighted requests, response headers, wildcard-with-credentials prohibition - https://fetch.spec.whatwg.org/#http-cors-protocol — CORS-safelisted methods/headers, preflight algorithm, non-wildcard credentialed response requirement +- Field reproduction 2026-09-02 (linkly-crew t2-live-visual-verify, Tauri/WKWebView app with a Node `collector.mjs`): the collector handled only `POST`; a full app rebuild and launch produced zero reports and `curl -X OPTIONS /report` returned 404. After adding an `OPTIONS` branch (204 with `access-control-allow-origin/methods/headers`) and the same headers on POST responses, the unchanged probe delivered 6 reports within 10 s diff --git a/wiki/backend/index.md b/wiki/backend/index.md index 7f86f69..51e737f 100644 --- a/wiki/backend/index.md +++ b/wiki/backend/index.md @@ -26,7 +26,7 @@ Match your situation to a "load when" line; load only matching pages. | [idempotency](common/api-design/idempotency.md) | An endpoint with side effects (create, charge, send) can receive the same request twice — client retry after timeout, user double-submit, gateway retry; designing idempotency-key storage; deciding which operations are safe to retry | | [pagination-contract](common/api-design/pagination-contract.md) | Designing a list endpoint's request/response contract — cursor vs page-number, limit caps, total counts, expired-cursor behavior (the backing SQL/index → databases/query-optimization/keyset-pagination) | | [unenforced-declarations](common/api-design/unenforced-declarations.md) | Your system accepts declarative input (config file, DSL/manifest, policy block, schema annotation) and part of what a caller may write is unimplemented — an unknown key, a verb outside your vocabulary, or a knob recorded but never acted on; a user reports "I declared X and nothing happened"; choosing between reject/warn/ignore and where that strictness is selected | -| [cors-and-preflight](common/api-design/cors-and-preflight.md) | A browser-based caller on a different origin fails with a CORS error in the console (a direct curl to the same endpoint works); deciding whether a change to an endpoint's method/headers/content-type will trigger an OPTIONS preflight; designing `Access-Control-*` headers for a credentialed vs public endpoint; allowlisting more than one origin | +| [cors-and-preflight](common/api-design/cors-and-preflight.md) | A browser-based caller on a different origin fails with a CORS error in the console (a direct curl to the same endpoint works); deciding whether a change to an endpoint's method/headers/content-type will trigger an OPTIONS preflight; designing `Access-Control-*` headers for a credentialed vs public endpoint; allowlisting more than one origin; an injected probe POSTs JSON to a hand-rolled local collector and nothing arrives | | [api-versioning-and-breaking-changes](common/api-design/api-versioning-and-breaking-changes.md) | An API has external callers you cannot enumerate or force-upgrade and you need to add/remove/rename/retype a field or endpoint; classifying a change as backward-compatible vs breaking; choosing a versioning mechanism (header vs URL); deprecating an old version (internal-only contract changes → [backend-common-change-impact-call-site-enumeration]) | | [agent-tool-granularity](common/api-design/agent-tool-granularity.md) | Adding agent capabilities to an app (MCP server, function-calling tools, plugin tools); choosing between exposing a workflow function and exposing primitives; an agent cannot perform an action the UI offers; reviewing a tool list for CRUD completeness, completion signaling, or UI/agent parity | diff --git a/wiki/infrastructure/agent-orchestration/control-signals-vs-primary-artifacts.md b/wiki/infrastructure/agent-orchestration/control-signals-vs-primary-artifacts.md index 7f739ff..28138b4 100644 --- a/wiki/infrastructure/agent-orchestration/control-signals-vs-primary-artifacts.md +++ b/wiki/infrastructure/agent-orchestration/control-signals-vs-primary-artifacts.md @@ -34,6 +34,7 @@ progress by running a script that writes outside its worktree. | "worker dead" | The substrate's own liveness call (runtime/task API, `tmux has-session -t `, container inspect) *plus* whether new commits appeared since the last check | | "worker produced file X" | `test -e`/`stat` on X | | "nothing happened yet" | Worker log mtime; an idle-looking worker mid-run and a worker that never started look identical from the status file | +| "this queued candidate is still pending" (an ingest/harvest queue row, a review-request ticket) | The destination store itself — grep it for the candidate's distinctive phrase. A producer that writes the store directly (a feature PR merging the same lesson, a manual edit) never touches the queue, so the row stays pending after the work has landed | 2. **Emit and consume status by an exact identifier the orchestrator assigned.** Have the worker write the run-scoped task id the orchestrator gave it, not an @@ -67,6 +68,7 @@ progress by running a script that writes outside its worktree. | An editor/Write tool succeeds where Bash was refused for the same path | The guardrail inspects Bash command text only, so the two channels disagree by design. Do not use the working channel to route around the rule — report it | | Heartbeat is fresh but no commits in N intervals | Fresh heartbeat plus unchanged primary artifact is the stalled state, distinct from alive-and-progressing and from dead; handle it as its own case | | The monitor is the only thing that can see the worker | Add the primary-artifact check to the monitor rather than trusting its verdict; a monitor with no artifact check cannot produce evidence | +| A queue row's content is already in the destination store (its distinctive phrase greps on `main`) | Retire the row as already-landed, recording the landing change's id (PR number, commit) in its outcome, and skip the processing round; a pending row nobody can promote re-crosses every batch threshold and re-spends the same research each run | | Several workers go quiet almost simultaneously: liveness checks pass, heartbeats stay fresh, diffs stop growing | Before restarting anything, search each worker's pane/terminal tail for the CLI's usage-limit marker (e.g. `You've hit your session limit · resets HH:MM`) — a usage-limit pause is idle waiting, not a crash, so every liveness probe passes. After the stated reset time, send a resume prompt that orders: re-verify state (`git status`, rerun the tests) → remaining definition-of-done → completion signal. A bare "continue" sent before the reset is consumed by the same limit message, and a resume without the state re-check makes the worker guess where it stopped | | The next task is dispatched to the same terminal immediately after the worker's done signal and fails runtime-unavailable | The done message is the worker's report time, not the substrate's release time — the CLI is still tearing down its stop-hook chain and the previous dispatch still occupies the terminal. Wait for the substrate's own idle signal (e.g. `orca terminal wait --for tui-idle`) before dispatching; and when the failed dispatch consumed the task, create a new task from the same spec — the consumed one cannot be retried | @@ -77,6 +79,7 @@ progress by running a script that writes outside its worktree. | Restart a worker because the watch script exited "dead" | Run the substrate liveness call and `git log` on its branch first | The verdict can come from an identifier the worker picked up from its environment, naming a different process entirely | | Let a worker resolve its own session/pane id for the status file | Pass the orchestrator-assigned task id into the worker and have it echo that back | Discovered identifiers are ambient state; assigned ids are the orchestrator's own namespace | | Trust a status script's silence as "signal sent" | `stat` the status file after the call and report the mtime | A gate-blocked command exits without writing and without printing | +| Research a queued candidate because its row says `pending` | Grep the destination store for its distinctive phrase first | The queue records intent at enqueue time; a direct write to the store after that leaves the row stale, and the row's status is a control signal about the store, not the store | | Retry a blocked status write with a rewritten path | Report the block and the rule id to the orchestrator | The rule is protecting the shared checkout; a workaround reintroduces exactly the corruption it prevents | ## Sources @@ -84,4 +87,5 @@ progress by running a script that writes outside its worktree. - https://man.openbsd.org/tmux — `has-session`, `display-message` behavior and session naming - https://code.claude.com/docs/en/hooks — hook architecture and path guardrails - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html — exit codes and output redirection +- Field evidence 2026-09-06 (dev-loop knowledge-flush run 20260906-003715): queue row f1ba9bf617fbd101 (graphify exit-0 handling) matched verbatim a row and a source line already on `main` in code-graph-as-orientation-layer.md, merged by PR #184 on 2026-09-04 from the same session that had harvested it; retired as dropped-already-merged with no wiki change - Field evidence 2026-08-06 (Claude CLI workers under tmux/Orca orchestration): three workers paused simultaneously on one usage-limit reset — identical `You've hit your session limit · resets 01:10` marker in each terminal tail while every liveness check passed; a state-recheck resume prompt sent after the reset resumed all three exactly at their interrupted step (test re-run). Same run: two dispatches issued immediately after `worker_done` both failed `runtime_unavailable` and consumed their tasks; dispatches issued after a `tui-idle` wait succeeded first try diff --git a/wiki/infrastructure/index.md b/wiki/infrastructure/index.md index 86c2ac3..26517ad 100644 --- a/wiki/infrastructure/index.md +++ b/wiki/infrastructure/index.md @@ -12,7 +12,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [control-signals-vs-primary-artifacts](agent-orchestration/control-signals-vs-primary-artifacts.md) | An orchestrator is about to restart, discard, merge, or keep waiting on a worker based on a status file, a watcher's exit code, or a heartbeat; a monitor reports a worker dead while it is committing; a worker's status write produced no output and you must decide whether it landed; distinguishing alive-and-progressing from stalled from dead; several workers went quiet at once while every liveness check passes (usage-limit pause); a dispatch issued right after a worker's done signal fails runtime-unavailable | +| [control-signals-vs-primary-artifacts](agent-orchestration/control-signals-vs-primary-artifacts.md) | An orchestrator is about to restart, discard, merge, or keep waiting on a worker based on a status file, a watcher's exit code, or a heartbeat; a monitor reports a worker dead while it is committing; a worker's status write produced no output and you must decide whether it landed; distinguishing alive-and-progressing from stalled from dead; a queued candidate's row still says pending while the destination store may already hold it; several workers went quiet at once while every liveness check passes (usage-limit pause); a dispatch issued right after a worker's done signal fails runtime-unavailable | | [shared-run-state](agent-orchestration/shared-run-state.md) | Several agent/worker sessions coordinate through files in one repository (status directory, briefs, escalations, claim files); choosing the path layout for that state; starting an orchestration in a repo that may already have one running; a watcher woke on a task id it did not create; the default branch moved during a run; a coordinator is about to reset a task's status file while re-delivering a prompt | | [pane-delivery-confirmation](agent-orchestration/pane-delivery-confirmation.md) | An orchestrator drives another program through a terminal multiplexer (`tmux send-keys` + `capture-pane`) and must decide whether the input was consumed, retry, or escalate; a pane diff is being used as delivery evidence; the target echoes but never runs the input; deciding *where* in a pane capture to search for a collapsed paste marker whose position depends on payload size, or what to report when the pane's input-box chrome cannot be located at all | | [session-completion-gates](agent-orchestration/session-completion-gates.md) | Writing a Stop/completion hook that blocks a worker session from ending while its phase is non-terminal; the gate fires on a worker that followed its own prompt; deciding the terminal phase set, the unknown-phase default, and how the gate bounds its own repetition; you are the worker the gate repeats on at an instructed pause and are deciding whether to advance your phase to silence it | diff --git a/wiki/mobile/security/sensitive-data-on-device.md b/wiki/mobile/security/sensitive-data-on-device.md index a4663cc..62bb9f3 100644 --- a/wiki/mobile/security/sensitive-data-on-device.md +++ b/wiki/mobile/security/sensitive-data-on-device.md @@ -14,7 +14,7 @@ sources: - https://developer.android.com/develop/ui/views/touch-and-input/copy-paste - https://developer.apple.com/documentation/uikit/uipasteboard last_verified: 2026-07-10 -related: [mobile-lifecycle-process-death-and-state, security-data-pii-handling, frontend-auth-token-handling-client-side] +related: [mobile-lifecycle-process-death-and-state, security-data-pii-handling, frontend-auth-token-handling-client-side, qa-environments-offscreen-render-capture-without-screen-recording] --- # Sensitive Data on Device — Storage Classes and Leak Surfaces diff --git a/wiki/qa/document-verification/editing-a-gated-document.md b/wiki/qa/document-verification/editing-a-gated-document.md index 9a9552a..8920fba 100644 --- a/wiki/qa/document-verification/editing-a-gated-document.md +++ b/wiki/qa/document-verification/editing-a-gated-document.md @@ -10,7 +10,7 @@ sources: - https://docs.vale.sh/checks/existence - https://github.com/DavidAnson/markdownlint/blob/main/doc/md013.md last_verified: 2026-08-14 -related: [qa-process-acceptance-criteria, qa-process-regression-scope, testing-quality-tests-that-cannot-fail] +related: [qa-process-acceptance-criteria, qa-process-regression-scope, testing-quality-tests-that-cannot-fail, qa-document-verification-model-coupled-guidance-aging-detector] --- # Editing a Document That Automated Text Gates Check diff --git a/wiki/qa/document-verification/model-coupled-guidance-aging-detector.md b/wiki/qa/document-verification/model-coupled-guidance-aging-detector.md new file mode 100644 index 0000000..ed9ff19 --- /dev/null +++ b/wiki/qa/document-verification/model-coupled-guidance-aging-detector.md @@ -0,0 +1,89 @@ +--- +id: qa-document-verification-model-coupled-guidance-aging-detector +domain: qa +category: document-verification +applies_to: [general, llm-docs, agent-wiki] +confidence: verified +sources: + - https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices + - https://github.com/choiyounggi/dev-loop/blob/main/scripts/wiki-lint-model-era.js + - https://github.com/choiyounggi/dev-loop/pull/178 +last_verified: 2026-09-06 +verified_model: claude-fable-5-1 +related: [qa-document-verification-spec-document-gates, qa-document-verification-editing-a-gated-document, testing-quality-guard-shape-vs-consequence, testing-quality-checks-that-cannot-pass, backend-common-llm-binding-instructions-for-agents] +--- + +# An Aging Detector for Documentation That Describes Model Behavior + +## When this applies + +You maintain guidance written for LLM agents — steering files (`CLAUDE.md`, +`AGENTS.md`), an agent-facing wiki, a prompt library, model-workaround notes — +and are designing the lint or report that surfaces pages whose advice may have +aged with a model generation; deciding what the detector matches on, how it +scopes the text it scans, and whether its verdict blocks CI. + +## Do this + +1. **Match on the subject of the prose, not on quirk phrasing.** The aging + surface is every page *about* model or harness behavior, and most of those + state directives without hedging: + +| Signal | Hit rate on a live 271-page agent wiki | Use it as | +|--------|----------------------------------------|-----------| +| Quirk phrases ("the model tends to…", "may hallucinate", "will over-trigger") | 3 / 271 pages | Nothing — it misses the surface it is meant to find | +| Subject keywords in prose (model names, `llm`, `subagent`, `context window`, `hallucinat`), each preceded by a non-word, non-hyphen boundary | 27 / 271 pages, concentrated where expected (`platforms/tools`, `backend/common/llm`) | The detector | + +2. **Scan prose only.** Skip frontmatter, the `## Sources` section, the URL part + of markdown links, and routing index files. A citation whose URL names a + model, or a page id such as `backend-common-llm-*`, references model-coupled + material without teaching from it; measured on the same corpus, a whole-file + grep for the keywords hits 32 pages where the scoped scan hits 21. +3. **Pair the detector with a frontmatter field that records the generation the + page was verified against** (`verified_model: `), and compute the + verdict from both: coupled **and** (field absent **or** its value contains no + current-generation token) → re-verification candidate. +4. **Make the current-generation list overridable** — command flag, then an + environment variable, then a built-in default — so the checker's own list can + age without editing every caller. +5. **Ship it report-only**: a distinct exit code for "candidates found" (the + script uses 3), the summary on stdout (`pages: N, model-coupled: M, + candidates: K`), one `revalidate:: ` line per candidate on + stderr, and a CI step that publishes the list without failing the build: + +| Case | Do | +|------|----| +| The live corpus already carries candidates on the day the check lands | Report-only, in its own non-blocking step; the semantic judgment (re-verify, rewrite, retire) stays with a human or the lint skill | +| You want a hard gate anyway | Gate a *diff-scoped* rule — a newly added model-coupled page must carry `verified_model` — and leave the corpus-wide count as a report | +| A candidate is re-verified and still correct | Set `verified_model` to the current generation and `last_verified` to today; the guidance did not change, its evidence date did | + +6. **Prove the detector on the live corpus before wiring it in**: count phrase + hits against subject hits, then read the flagged list — concentration in the + directories that discuss models and harnesses is the evidence it measures + aging rather than vocabulary ([testing-quality-checks-that-cannot-pass] for + authoring a check whose target may not exist yet). + +## Edge cases + +| Case | Then | +|------|------| +| A page mentions a hyphen-joined id containing a keyword (`backend-common-llm-context-window-budget`) and no model behavior | The leading boundary `(? `DEV_LOOP_CURRENT_MODELS` > default; exit 0/3/4, report-only by design +- https://github.com/choiyounggi/dev-loop/pull/178 — design rationale and measurements: "explicit 'model tends/may/will' phrasing appears on 3/271 pages while model-coupled subjects cover ~27/271, concentrated in platforms/tools and backend/common/llm"; live run `pages: 275, model-coupled: 20, candidates: 20`; "Deliberately NOT in the blocking CI wiki-checks step" +- Local reproduction 2026-09-06 (dev-loop `main` at 1.21.0, 276 pages): `node scripts/wiki-lint-model-era.js wiki` → `pages: 276, model-coupled: 21, candidates: 21`, exit 3, flagged pages under `platforms/tools`, `infrastructure/agent-orchestration`, `security/agent-exposure`, `backend/common/llm`; a whole-file `grep -liE` for the same keywords over the same 276 pages hit 32 diff --git a/wiki/qa/environments/offscreen-render-capture-without-screen-recording.md b/wiki/qa/environments/offscreen-render-capture-without-screen-recording.md new file mode 100644 index 0000000..bd5b591 --- /dev/null +++ b/wiki/qa/environments/offscreen-render-capture-without-screen-recording.md @@ -0,0 +1,84 @@ +--- +id: qa-environments-offscreen-render-capture-without-screen-recording +domain: qa +category: environments +applies_to: [macos, spritekit, appkit] +confidence: verified +sources: + - https://developer.apple.com/documentation/spritekit/skview/texture(from:) + - https://developer.apple.com/documentation/spritekit/sktexture/cgimage() + - https://developer.apple.com/documentation/spritekit/skrenderer + - https://developer.apple.com/documentation/imageio/cgimagedestinationcreatewithurl(_:_:_:_:) + - https://developer.apple.com/documentation/imageio/kcgimagepropertygifdelaytime + - https://developer.apple.com/documentation/appkit/nsview/cachedisplay(in:to:) + - https://developer.apple.com/videos/play/wwdc2019/701/ + - https://support.apple.com/guide/security/protecting-app-access-to-user-data-secc01781f46/web + - https://developer.apple.com/documentation/macos-release-notes/macos-15_1-release-notes +last_verified: 2026-09-06 +related: [qa-environments-browser-console-capture-gaps, qa-process-completion-claims, qa-bug-reports-reproducible-reports, mobile-security-sensitive-data-on-device] +--- + +# Screenshots and Play Reels of a macOS App Without Screen Recording Permission + +## When this applies + +A screenshot or a short interaction reel of a macOS AppKit/SpriteKit app is +needed for a README, a bug report, or a CI artifact, and the capturing process +has no Screen Recording permission — `screencapture` fails, or the host is +headless/CI with nobody to click the grant. Also when choosing how to build an +animated GIF from frames without adding a tool dependency. + +## Do this + +1. **Render the app's own content off-screen instead of reading the screen.** + Screen Recording (TCC) gates reading the screen buffer — since macOS + Catalina "the user must use the security and privacy preference pane to + preapprove apps to record the entire screen or the contents of windows other + than their own", while "Apps can freely record the contents of their own + windows". An app drawing itself into an in-memory image needs no grant. + +| Case | Do | +|------|----| +| Still image of a SpriteKit scene | `SKView.texture(from: node)` (or `texture(from:crop:)`), then `SKTexture.cgImage()`, then write it with `CGImageDestinationCreateWithURL` + `CGImageDestinationAddImage` as PNG. The node "does not need to appear in the view's presented scene", so a staged arrangement can be composed off to the side | +| Animated reel | Capture frames on a fixed cadence into one `CGImageDestination` of type `UTType.gif`; pass each frame a `kCGImagePropertyGIFDictionary` with `kCGImagePropertyGIFDelayTime` (seconds) and set `kCGImagePropertyGIFLoopCount` on the destination — ImageIO ships with the SDK | +| Choosing the cadence | Capture at 10 fps (0.1 s delay): the delay time is "clamped to a minimum of 100 milliseconds", so a 15 fps capture plays back at 10 fps, 1.5× slower than real time | +| Deterministic staged scene (feature showcase) | Build the exact node arrangement in code, step the scene yourself, capture — the same frame every run | +| Real play reel | Poll the scene tree each tick, drive the same input handlers a user would (a bot), and capture one frame per tick | +| AppKit view with no SpriteKit | `bitmapImageRepForCachingDisplay(in:)` then `cacheDisplay(in:to:)` — it "draws the specified area of the view, and its descendants, into a provided bitmap-representation object" — then encode via ImageIO as above | +| The deliverable must show the whole screen or another app's window | That is screen capture: grant Screen Recording in System Settings › Privacy & Security, relaunch the capturing process, and use ScreenCaptureKit (`CGWindowListCreateImage` is deprecated, and macOS 15.1 adds "enhanced user awareness" dialogs for the deprecated capture APIs) | + +2. **Prove one frame before batching.** Write the first frame, check its pixel + size against the expected scene size and that it is not blank, then run the + loop — a wrong view size or an unattached view produces a full set of empty + frames that looks like a success at the file-count level + ([qa-process-completion-claims]). + +## Edge cases + +| Case | Then | +|------|------| +| `screencapture` prints `could not create image` | The missing Screen Recording grant, not a display fault (field evidence, desk-bat 2026-08-19); switch to the off-screen path when the grant cannot be given, and report the permission state beside the capture in a bug report ([qa-bug-reports-reproducible-reports]) | +| Retina output size | The texture size follows the node's accumulated frame in scene points; set the `SKView` frame and the scene size explicitly before capturing and read the written image's pixel dimensions back in step 2 | +| The view was never attached to a window | Apple documents only that the *node* need not be in the presented scene; the field run rendered from the live app's `SKView`. Run step 2 first and fall back to `SKRenderer` (Metal, "renders a scene into a custom Metal rendering pipeline") when the frame is blank | +| The reel must show sensitive data (balances, tokens) | Stage the scene with fixture values; a committed GIF is a published artifact ([mobile-security-sensitive-data-on-device]) | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Run `screencapture` or `CGWindowListCreateImage` on a host without Screen Recording permission | Render off-screen (`SKView.texture(from:)`, `SKRenderer`, `NSView.cacheDisplay`) and encode with ImageIO | The grant needs a user in System Settings and a process relaunch (field evidence); the app's own content was never gated | +| Install ffmpeg to assemble the GIF | `CGImageDestination` with `UTType.gif`, per-frame `kCGImagePropertyGIFDelayTime`, `kCGImagePropertyGIFLoopCount` | ImageIO is in the SDK and supports per-frame delay and loop count directly | +| Capture at 15–30 fps for a smoother GIF | 10 fps, 0.1 s per frame | Delays under 100 ms are clamped up, so higher capture rates only slow playback | + +## Sources + +- https://developer.apple.com/documentation/spritekit/skview/texture(from:) — "Renders the contents of a node tree and returns the rendered image as a texture"; "The node being rendered does not need to appear in the view's presented scene" +- https://developer.apple.com/documentation/spritekit/sktexture/cgimage() — "Returns the texture's image data as a Quartz 2D image" +- https://developer.apple.com/documentation/spritekit/skrenderer — "An object that renders a scene into a custom Metal rendering pipeline and drives the scene update cycle" +- https://developer.apple.com/documentation/imageio/cgimagedestinationcreatewithurl(_:_:_:_:) — "Creates an image destination that writes image data to the specified URL"; `CGImageDestinationAddImage` "Adds an image to an image destination" +- https://developer.apple.com/documentation/imageio/kcgimagepropertygifdelaytime — "The number of seconds to wait before displaying the next image in an animated sequence, clamped to a minimum of 100 milliseconds"; `kCGImagePropertyGIFLoopCount` — "The number of times to repeat an animated sequence" +- https://developer.apple.com/documentation/appkit/nsview/cachedisplay(in:to:) — "Draws the specified area of the view, and its descendants, into a provided bitmap-representation object" +- https://developer.apple.com/videos/play/wwdc2019/701/ — WWDC19 "Advances in macOS Security": Catalina gates recording "the entire screen or the contents of windows other than their own" behind a user pre-approval; "Apps can freely record the contents of their own windows" +- https://support.apple.com/guide/security/protecting-app-access-to-user-data-secc01781f46/web — "Screen recording (for example, static screen shots and video)" is a TCC-protected category configured in System Settings +- https://developer.apple.com/documentation/macos-release-notes/macos-15_1-release-notes — "Applications using our deprecated content capture technologies now have enhanced user awareness policies"; `CGWindowListCreateImage` is marked deprecated in Apple's documentation +- Field evidence 2026-08-19 (desk-bat, SpriteKit macOS app, commit d0da30d): `screencapture` failed with `could not create image` under a missing Screen Recording grant; `SKView.texture(from:)` frames plus an ImageIO GIF destination produced 3 screenshots and a 203-frame, 15 fps, 124 KB GIF committed to the README, with gameplay driven by a scene-tree-polling bot diff --git a/wiki/qa/environments/test-environment-parity.md b/wiki/qa/environments/test-environment-parity.md index e5d92f0..59e493f 100644 --- a/wiki/qa/environments/test-environment-parity.md +++ b/wiki/qa/environments/test-environment-parity.md @@ -7,7 +7,7 @@ confidence: field-tested sources: - https://12factor.net/dev-prod-parity last_verified: 2026-07-10 -related: [qa-process-release-gates, qa-process-post-release-verification, qa-bug-reports-reproducible-reports, testing-strategy-orm-generated-test-schema] +related: [qa-process-release-gates, qa-process-post-release-verification, qa-bug-reports-reproducible-reports, testing-strategy-orm-generated-test-schema, testing-data-testcontainers-reaper-on-docker-desktop-macos] --- # Knowing What a Staging Pass Actually Clears diff --git a/wiki/qa/index.md b/wiki/qa/index.md index 18a4e9a..bd04ff2 100644 --- a/wiki/qa/index.md +++ b/wiki/qa/index.md @@ -42,6 +42,7 @@ Match your situation to a "load when" line; load only matching pages. | [generated-reference-drift-gates](document-verification/generated-reference-drift-gates.md) | Writing or reviewing the reference material that enumerates a closed vocabulary an agent will emit tokens from (DSL verbs, config keys, diagnostic codes, enum members) in a plugin/skill/SDK; deciding whether to hand-write that list or generate it from the owning constant; choosing what gates a generated document beyond a `--check` diff; a documented token compiles to a silent no-op instead of an error | | [retiring-a-provisional-marker](document-verification/retiring-a-provisional-marker.md) | Turning provisional markers (`[추정]`, TBD, "assumed", DRAFT) into settled statements in an ADR/RFC/spec that has been through review rounds, so the document also carries a review checklist and a round history referring to those markers; a checklist row stayed `[x]` on evidence you just deleted; a coordinator's marker count and yours disagree while you read the same file | | [editing-a-gated-document](document-verification/editing-a-gated-document.md) | Editing or rewording a document that grep/regex gates or a lint config check; a gate fails on wording whose meaning did not change; reflowing prose a test asserts as a verbatim phrase (CI red on one platform only); describing what an upstream spec says without tripping a "do not redefine it" gate; a check matches the pattern your own document quotes; recording an audit verdict inside the document that was audited; deciding which checks to re-run after editing a gated document | +| [model-coupled-guidance-aging-detector](document-verification/model-coupled-guidance-aging-detector.md) | Designing a lint or report that surfaces pages of agent-facing guidance (steering files, an agent wiki, prompt libraries) whose advice may have aged with a model generation; choosing what such a detector matches on (subject keywords vs quirk phrases), which text it scans (prose vs citations/URLs/index files), and whether its verdict blocks CI; adding a `verified_model`-style field and deciding how its absence is reported | ## environments @@ -50,6 +51,7 @@ Match your situation to a "load when" line; load only matching pages. | [test-environment-parity](environments/test-environment-parity.md) | A bug reproduces only in production; planning what a staging environment must mirror; deciding whether a staging pass clears a release | | [headless-browser-bot-blocking](environments/headless-browser-bot-blocking.md) | QA/dogfooding an external production site through a headless browser shows an intact page shell but empty lists/data with generic "temporary delay" toasts; deciding between "their server is down" and "our client is classified as a bot"; data APIs alone return 4xx while static assets load | | [browser-console-capture-gaps](environments/browser-console-capture-gaps.md) | About to read a browser-automation tool's console output as a QA verdict ("no errors", "the script never ran"); the collected list is empty for a page known to log on load; deciding when the collector attaches and whether to clear+reload; judging whether a browser extension's content script executed when its logs never appear | +| [offscreen-render-capture-without-screen-recording](environments/offscreen-render-capture-without-screen-recording.md) | A screenshot or play reel of a macOS AppKit/SpriteKit app is needed for docs, a bug report, or CI and the host has no Screen Recording (TCC) grant (`screencapture` fails with `could not create image`, or the machine is headless); building an animated GIF from rendered frames without ffmpeg; choosing the GIF frame cadence | ## bug-reports diff --git a/wiki/security/index.md b/wiki/security/index.md index 3066214..a07652a 100644 --- a/wiki/security/index.md +++ b/wiki/security/index.md @@ -47,6 +47,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| | [secrets-in-code](secrets/secrets-in-code.md) | Code needs an API key/password/private key; a secret was just committed/pasted/logged (leak response, rotation); setting up repo secret hygiene (scanners, env files); deciding whether a value may ship in client code (CI/build secrets → infrastructure/ci-cd/secrets-handling) | +| [ciphertext-orphaned-by-a-regenerated-key](secrets/ciphertext-orphaned-by-a-regenerated-key.md) | Rows encrypted at rest (AES-GCM or similar) stop decrypting while the service logs that the key loaded, and an env-init/bootstrap script can regenerate the key; deciding what to check before deleting unreadable rows; storing a key id/version with ciphertext; making key generation in a setup script idempotent | ## dependencies diff --git a/wiki/security/secrets/ciphertext-orphaned-by-a-regenerated-key.md b/wiki/security/secrets/ciphertext-orphaned-by-a-regenerated-key.md new file mode 100644 index 0000000..ca868dc --- /dev/null +++ b/wiki/security/secrets/ciphertext-orphaned-by-a-regenerated-key.md @@ -0,0 +1,83 @@ +--- +id: security-secrets-ciphertext-orphaned-by-a-regenerated-key +domain: security +category: secrets +applies_to: [general, aes-gcm, node] +confidence: verified +sources: + - https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html + - https://nodejs.org/api/crypto.html + - https://guides.rubyonrails.org/active_record_encryption.html + - https://developers.google.com/tink/design/keysets +last_verified: 2026-09-06 +related: [security-secrets-secrets-in-code, debugging-methodology-hypothesis-testing, debugging-methodology-probe-path-vs-operation-path, infrastructure-data-backup-and-restore] +--- + +# Stored Ciphertext That No Longer Decrypts After an Environment Script Regenerated the Key + +## When this applies + +Rows encrypted server-side with a symmetric AEAD (AES-GCM or similar) fail to +decrypt although they were readable before; the service logs that the key +loaded; and an environment bootstrap script (`init:env`, a setup task, a +container entrypoint) is able to create or overwrite the key. Also when you are +about to delete the unreadable rows. + +## Do this + +1. **Separate the two claims the logs conflate.** "Key loaded" is a statement + about the process; "this row was encrypted under the loaded key" is a + statement about the data. A wrong key and a tampered ciphertext produce the + same failure — in Node, `decipher.final()` "will throw, indicating that the + cipher text should be discarded due to failed authentication" — so no error + text distinguishes them. +2. **Run one fresh round-trip.** Encrypt a sentinel with the loaded key and + decrypt it in the same process. Success proves the code path and the key; + the fault is then in the relationship between key and stored data. +3. **Scan read-only before any deletion.** Attempt decryption of every affected + row with the loaded key (and with every previous key you still hold) and + record `recoverable / total`; act on the count: + +| Finding | Do | +|---------|----| +| The key fails to load (missing variable, provider error) | Fix the load path; the data has not been touched and needs nothing | +| Key loads, round-trip passes, `0 / N` rows recoverable | Every row predates the current key — a regeneration happened. Recover the previous key (below) or, for disposable development data, delete the rows and record the count in the change note | +| Key loads, a subset fails | A partial rotation: group failures by the stored key id/version; decrypt each group with its own key; treat rows whose key is gone as the `0 / N` case | +| Rows carry no key id or version at all | Add one now (a version column, or a key-id prefix on the ciphertext) before re-encrypting, so the next mismatch reads as "unknown key id" at the row instead of an authentication failure | + +4. **Recover before you re-encrypt.** Old keys are retained precisely for this: + OWASP notes "old keys should … be stored for a certain period after they have + been retired, in case old backups of copies of the data need to be + decrypted". Read the previous value from the secret manager's version history + or a backup ([infrastructure-data-backup-and-restore]), decrypt, re-encrypt + under the current key, then retire the old key. +5. **Make the bootstrap idempotent.** Generate the key only when none exists, and + make any path that would overwrite an existing key stop with a loud message + naming the rows that depend on it; an unintended regeneration is an unplanned + rotation with no re-encryption step. + +## Edge cases + +| Case | Then | +|------|------| +| The data is production data | Deletion is not an option at any count; recover the key from the secret store's version history or backups, re-encrypt, and log the incident | +| A rotation is in progress by design | Store a key identifier with each ciphertext and decrypt by it — Rails ActiveRecord Encryption keeps `previous:` key providers for exactly this, and Tink prefixes "ciphertexts with a 5-byte string derived from the ID" so the right key is selected without trying them all | +| The round-trip of step 2 fails too | The fault is the key material or the code, not the data — go to [debugging-methodology-hypothesis-testing] with "wrong key bytes" and "wrong IV/tag handling" as the competing suspects | +| The "key loaded" log line is your only evidence the key is right | That is a probe of a different path than the failing operation ([debugging-methodology-probe-path-vs-operation-path]); the round-trip plus the row scan are the operation-path probes | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Debug the decrypt code because the key loads fine | Run the sentinel round-trip, then the read-only row scan | A passing round-trip beside a failing stored row locates the fault in key-to-data lineage, not in code | +| Delete rows on the first decryption error | Scan every row read-only and record `recoverable / total` first | A partial count means recoverable data exists; the count is also the evidence for the deletion decision | +| Let the init script regenerate the key on every run | Generate only when absent, warn loudly on overwrite | OWASP: "the code and processes required to rotate a key are in place before they are required" — rotation is a deliberate procedure that includes re-encryption | +| Keep a single key with no identifier on the ciphertext | Store a key version/id with each row | The next mismatch then fails as a lookup at the row, not as an authentication error indistinguishable from tampering | + +## Sources + +- https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html — key rotation by "Decrypting it and re-encrypting it with the new key" or by "Marking each item with the ID of the key that was used to encrypt it, and storing multiple keys to allow the old data to be decrypted"; "old keys should … be stored for a certain period after they have been retired, in case old backups of copies of the data need to be decrypted"; "the code and processes required to rotate a key are in place before they are required" +- https://nodejs.org/api/crypto.html — for authenticated modes, when the tag does not verify "decipher.final() will throw, indicating that the cipher text should be discarded due to failed authentication" — a wrong key surfaces as this same error +- https://guides.rubyonrails.org/active_record_encryption.html — `config.active_record.encryption.previous = [ { key_provider: MyOldKeyProvider.new } ]` and the per-attribute `previous:` option: previous keys are kept so existing ciphertext stays readable through a rotation +- https://developers.google.com/tink/design/keysets — Tink can "prefix ciphertexts with a 5-byte string derived from the ID" of the key so decryption selects the key instead of trying every key in the keyset +- Field reproduction 2026-08-30 (linkly-calendar, `ChatEncryptionService`, AES-GCM, Node): the log showed "environment loaded (true)" and "decryption failed (keyVersion=1)" at the same second after `init:env` had been re-run and had regenerated the key; a read-only scan found 124/124 rows unrecoverable with the loaded key; the development rows were deleted and the service returned to normal diff --git a/wiki/security/secrets/secrets-in-code.md b/wiki/security/secrets/secrets-in-code.md index db2c8f7..bc5f72f 100644 --- a/wiki/security/secrets/secrets-in-code.md +++ b/wiki/security/secrets/secrets-in-code.md @@ -10,7 +10,7 @@ sources: - https://nextjs.org/docs/pages/building-your-application/configuring/environment-variables - https://vite.dev/guide/env-and-mode last_verified: 2026-07-12 -related: [infrastructure-ci-cd-secrets-handling] +related: [infrastructure-ci-cd-secrets-handling, security-secrets-ciphertext-orphaned-by-a-regenerated-key] --- # Keeping Secrets Out of Code and Responding to a Leak diff --git a/wiki/testing/data/testcontainers-reaper-on-docker-desktop-macos.md b/wiki/testing/data/testcontainers-reaper-on-docker-desktop-macos.md new file mode 100644 index 0000000..c77b7e2 --- /dev/null +++ b/wiki/testing/data/testcontainers-reaper-on-docker-desktop-macos.md @@ -0,0 +1,81 @@ +--- +id: testing-data-testcontainers-reaper-on-docker-desktop-macos +domain: testing +category: data +applies_to: [testcontainers, python, docker-desktop, macos] +confidence: verified +sources: + - https://github.com/testcontainers/testcontainers-python/blob/main/README.md + - https://java.testcontainers.org/features/configuration/ + - https://golang.testcontainers.org/features/configuration/ + - https://golang.testcontainers.org/system_requirements/rancher/ + - https://docs.rancherdesktop.io/how-to-guides/using-testcontainers/ + - https://docs.docker.com/desktop/settings-and-maintenance/settings/ + - https://github.com/testcontainers/testcontainers-java/issues/8170 + - https://github.com/testcontainers/testcontainers-java/issues/7678 + - https://github.com/testcontainers/testcontainers-go/issues/399 +last_verified: 2026-09-06 +related: [testing-data-test-data-and-isolation, qa-environments-test-environment-parity, debugging-signals-reading-error-messages] +--- + +# Testcontainers' Reaper Cannot Mount the Docker Socket on Docker Desktop for macOS + +## When this applies + +A Testcontainers suite (testcontainers-python here; the Java and Go libraries +share the mechanism) on a macOS developer machine with Docker Desktop stops +before any test runs — `0 tests`, `errors=N` — with an error about mounting +`~/.docker/run/docker.sock` or `/host_mnt/Users//.docker/run/docker.sock` +(HTTP 500, `operation not supported`), while the identical suite is green on a +Linux CI runner. + +## Do this + +1. **Read it as a host-environment fault, not a code fault.** Ryuk, the cleanup + sidecar, bind-mounts the Docker socket into its own container; Docker + Desktop's per-user socket path lives on the host, and the VM that runs the + daemon cannot mount it. The containers under test never started — the + failure is upstream of every test body. +2. **Pick the fix by what you can change on the host**, in this order: + +| Case | Do | +|------|----| +| Docker Desktop settings are yours to change | Settings › Advanced › enable **"Allow the default Docker socket to be used"** — it "Creates `/var/run/docker.sock`", the path Ryuk mounts by default; restart Docker Desktop and rerun with Ryuk on | +| The setting cannot be enabled (managed machine) | `export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock` — the socket path *inside the VM*, which is where the mount is resolved; the same override the Rancher Desktop and Colima guides prescribe. Ryuk stays on | +| Neither is possible | `export TESTCONTAINERS_RYUK_DISABLED=true` on the dev host only, and remove leftover containers/networks/volumes yourself after interrupted runs — the library warns this "will prevent testcontainers from automatically cleaning up resources, which is particularly important in tests which timeout" | +| Linux CI runner (`ubuntu-latest`) | Change nothing — `/var/run/docker.sock` is the real daemon socket there, and the suite already passes | + +3. **Keep the workaround out of shared config.** Put the variable in the + developer's shell profile or an untracked local env file, and document it in + the README as a macOS Docker Desktop note; committing it into `pytest.ini`, + `conftest.py`, or the CI workflow disables cleanup on every machine that did + not have the problem. + +## Edge cases + +| Case | Then | +|------|------| +| Colima, Rancher Desktop, or OrbStack instead of Docker Desktop | Same override mechanism: the vendor guides set `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock` (Rancher Desktop also sets `TESTCONTAINERS_HOST_OVERRIDE` to the VM's address) | +| A specific container class still mounts the wrong socket after the override | The class binds the socket path before host detection resolves it (testcontainers-java#7678, LocalStack; fixed upstream) — upgrade the library before changing more host settings | +| Ryuk is disabled and a run was interrupted | `docker ps -a --filter label=org.testcontainers` lists what the reaper would have removed; delete those containers and their networks/volumes before the next run | +| The error text names `/host_mnt/Users/...` | Docker Desktop's file-sharing mount of the host socket path — the socket cannot be shared into the VM as a file (testcontainers-java#8170, closed as environment); it is the same fault as the `~/.docker/run/docker.sock` message | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Mock the database or rewrite the suite because "Docker is broken on macOS" | Fix the socket path (Desktop setting or `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE`) | Only Ryuk's mount failed; the containers under test start once the reaper can | +| Commit `TESTCONTAINERS_RYUK_DISABLED=true` into the repo's test or CI config | Scope it to the affected dev host and document it | Ryuk removes containers, networks, volumes and images after a run; disabling it everywhere leaks resources on every machine and on CI | +| Point the override at `~/.docker/run/docker.sock` | Point it at `/var/run/docker.sock` | The override names the path the *daemon* mounts into Ryuk, which resolves inside the VM — the host-side per-user path is the one that fails | + +## Sources + +- https://github.com/testcontainers/testcontainers-python/blob/main/README.md — configuration table: `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE` (default `/var/run/docker.sock`, "Path to Docker's socket used by ryuk"), `TESTCONTAINERS_RYUK_DISABLED` (default `false`, "Disable ryuk"), runtime equivalent `testcontainers_config.ryuk_docker_socket` +- https://java.testcontainers.org/features/configuration/ — `TESTCONTAINERS_RYUK_DISABLED`: "If your environment already implements automatic cleanup of containers after the execution, but does not allow starting privileged containers, you can turn off the Ryuk container"; `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE`: "Path to Docker's socket. Used by Ryuk, Docker Compose, and a few other containers that need to perform Docker actions" +- https://golang.testcontainers.org/features/configuration/ — disabling Ryuk "will prevent testcontainers from automatically cleaning up resources, which is particularly important in tests which timeout as they don't run test clean up" +- https://golang.testcontainers.org/system_requirements/rancher/ and https://docs.rancherdesktop.io/how-to-guides/using-testcontainers/ — `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock` for a macOS VM-based Docker runtime +- https://docs.docker.com/desktop/settings-and-maintenance/settings/ — "Allow the default Docker socket to be used … Creates /var/run/docker.sock which some third party clients may use to communicate with Docker Desktop" +- https://github.com/testcontainers/testcontainers-java/issues/8170 — `error while creating mount source path '/host_mnt/Users/_user/.docker/run/docker.sock' … operation not supported` on Docker Desktop for Mac; closed as environment +- https://github.com/testcontainers/testcontainers-java/issues/7678 — a container fails to start when "Allow the default Docker socket to be used" is unchecked; the class bound the socket before host detection +- https://github.com/testcontainers/testcontainers-go/issues/399 — the socket override was added for non-standard socket paths on macOS VM runtimes +- Field reproduction 2026-08-30 (linkly, testcontainers-python, macOS Docker Desktop): the suite exited rc=5 with `errors=2, 0 tests` on the `~/.docker/run/docker.sock` mount error twice; with `TESTCONTAINERS_RYUK_DISABLED=true` it ran 27/27 green twice; GitHub Actions `ubuntu-latest` ran the same suite green with no variable set (run 33309041474) diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 4e65ac9..1bc9eff 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -22,6 +22,7 @@ Match your situation to a "load when" line; load only matching pages. | [orm-generated-test-schema](strategy/orm-generated-test-schema.md) | Planning a regression test for a defect caused by the ORM model and the real database disagreeing (nullability, type, length, index); checking what a test profile's `ddl-auto`/schema source makes reproducible; choosing between a migration-built test database with `validate` and an `information_schema` comparison gate; reviewing a plan that promises to "add a test that reproduces" a drift-shaped defect | | [failing-test-first](strategy/failing-test-first.md) | Implementing new behavior or a bug fix and deciding the order of test and production code; a test passed on its first-ever run; production code exists that no test required; expected values are about to be copied from the code's own output; encoding a bug repro as a regression test | | [cross-layer-effect-tests](strategy/cross-layer-effect-tests.md) | About to mark a change tested when its tests exercise only the changed unit; the changed path fires callbacks/middleware/hooks/jobs/DB triggers; the change persists state in more than one step; several surfaces (API, CLI, job) expose the same changed logic; deciding whether unit-level coverage suffices for a change | +| [executable-target-tests-in-swiftpm](strategy/executable-target-tests-in-swiftpm.md) | A SwiftPM `.executableTarget` (macOS app or CLI without an Xcode project) has internal types that need XCTest coverage and you are about to skip the tests or split out a library because "an executable target cannot be imported"; deciding the tools-version, `swift test` flags, or when a library split is still justified | ## quality @@ -38,7 +39,7 @@ Match your situation to a "load when" line; load only matching pages. | [signed-link-verification-assertions](quality/signed-link-verification-assertions.md) | Choosing the assertions for code that builds a URL carrying its own credential (approval or magic link with `?t=`, object-storage presigned URL, webhook callback, unsubscribe link); such a test is green while the link the product sends is rejected by its receiver; deciding between asserting the parameter's presence, running the token through the production verifier, and issuing the real request; pinning a token to both its key and its subject | | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate; a value-preserving move of a literal into config needs a test that detects re-inlining without asserting an internal | -| [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect; a shell-test (bats) assertion passes mid-test on code it should fail | +| [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect; a shell-test (bats) assertion passes mid-test on code it should fail; a test re-asserts the literal a layout constant is declared as | | [checks-that-cannot-pass](quality/checks-that-cannot-pass.md) | Authoring a check whose target does not exist yet (grep/regex gate on an unwritten file or doc section, lint/scan rule, schema assertion on an unbuilt endpoint, a plan's verification command) and it has only ever been observed failing; reviewing a plan's gates before adopting them; separating "target missing" from "content missing" in a gate's exit status. This owns unwritten-target gate patterns for spec/RFC documents too — qa/document-verification does not | | [spec-artifact-checks](quality/spec-artifact-checks.md) | Authoring or reviewing the check itself: that a mapping table covers every rule/field/enum case, that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo (deciding whether a passing gate is enough to *accept the deliverable* → wiki/qa/document-verification/spec-document-gates.md) | | [schema-additions-under-a-golden-gate](quality/schema-additions-under-a-golden-gate.md) | Adding a node kind, variant, discriminator value, or field to a document format (IR, JSON Schema, spec artifact) whose only automated gate builds its negatives by mutating one committed golden example; the gate or the whole suite comes back green right after a schema change; deciding which negative each new schema keyword needs, and whether a green suite that never loads the schema is evidence at all | @@ -62,6 +63,7 @@ Match your situation to a "load when" line; load only matching pages. | [artifact-leakage-from-a-suite](data/artifact-leakage-from-a-suite.md) | Temp directories, build outputs, or scratch files pile up in the repo or system temp after a suite runs; a clone grows with no obvious owner; you suspect the leak comes from everywhere and need a way to locate it; deciding between per-site cleanup, the runner's owned-temp API, and a static rule that enforces the convention | | [harness-vs-run-path-fixtures](data/harness-vs-run-path-fixtures.md) | A test/spec harness and the production entry point each synthesize the program's input and the harness reads a narrower declaration set; a harness run reports a guarded step skipped with exit 0 and you are about to record that skip as program behavior; deciding whether a false guard means a false operand or an absent one | | [test-data-and-isolation](data/test-data-and-isolation.md) | Tests need fixture data and you are choosing how to create it; tests pass alone but fail together (or vice versa); DB cleanup, shared fixtures, time-dependent logic, or unique-value collisions; a suite behaves differently inside a harness-spawned session that injects env vars | +| [testcontainers-reaper-on-docker-desktop-macos](data/testcontainers-reaper-on-docker-desktop-macos.md) | A Testcontainers suite on macOS Docker Desktop exits with `0 tests` on an error about mounting `~/.docker/run/docker.sock` (or `/host_mnt/...docker.sock`) while the same suite is green on Linux CI; choosing between the Desktop default-socket setting, `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE`, and disabling Ryuk, and where such a workaround may live | ## mocking diff --git a/wiki/testing/quality/guard-shape-vs-consequence.md b/wiki/testing/quality/guard-shape-vs-consequence.md index ecf1713..cabecd0 100644 --- a/wiki/testing/quality/guard-shape-vs-consequence.md +++ b/wiki/testing/quality/guard-shape-vs-consequence.md @@ -8,7 +8,7 @@ sources: - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html - https://pitest.org/ last_verified: 2026-08-06 -related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope, testing-quality-source-text-wiring-assertions] +related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope, testing-quality-source-text-wiring-assertions, qa-document-verification-model-coupled-guidance-aging-detector] --- # A Repo-Wide Guard That Fires on a Legitimate Artifact diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index fe33077..faa1115 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -75,6 +75,7 @@ suite reported as covered, or you are auditing a suspiciously green suite. | Testing the mock instead of the code (mock returns X, test asserts X came back) | Assert the unit's transformation of its inputs, not the pass-through; when no transformation exists at this layer, test the layer that has one ([testing-mocking-what-to-mock]) | | Copied test body with the name changed but identical inputs and expectation | Give each case distinct inputs and its own expectation; delete exact duplicates — a renamed copy re-proves the same fact and guards nothing new | | Assertion inherited from a shared base class, mixin, or parameterised harness, whose name announces the new subject's whole shape while its body pins the original narrow scope | Read the inherited body and list what it compares; add a subject-specific assertion for each part of the shape the name claims, then prove each one with its own mutation | +| Assertion restates the literal a constant is itself declared as (`XCTAssertEqual(TabBar.totalInsetHeight, 96)` while the declaration reads `totalInsetHeight = 96`) — the geometry the number documents is composed elsewhere | Declare the constant as the sum of the components it describes (`barHeight + bottomPadding + …`) and assert a relation between production values — constant against the measured frame, or constant against the sum of its named parts. Both sides of a literal re-assertion come from the same line, so it stays green while the real layout drifts to another value | | Bats assertion written as `[[ … ]]` anywhere but the test's last command, when bats resolves to bash 3.2 (macOS system bash) — a false `[[ ]]` mid-test does not fail the test | Write bats assertions as simple commands — `[ … ]` or `printf '%s\n' "$output" \| grep -qF "expected"` — which fail at any position; before bash 4.0, `set -e` ignores a failing compound command, so a mid-test `[[ ]]` is decoration on that shell (same shape as the documented bats `!`-negation gotcha) | 6. **Coverage note:** a covered line is only an executed line. Use coverage to @@ -132,4 +133,5 @@ suite reported as covered, or you are auditing a suspiciously green suite. - https://bats-core.readthedocs.io/en/stable/gotchas.html, https://www.shellcheck.net/wiki/SC2314 — the documented same-shape gotcha: bats commands whose failure is excluded from errexit (negated `!` commands) "can never fail when used in the middle of a test" - Local reproduction 2026-08-06 (Bats 1.14.0, GNU bash 3.2.57, macOS arm64): a false `[[ "a" == *"zzz"* ]]` mid-test → `ok`; the same false comparison as `[ "a" = "zzz" ]` or piped `grep -qF` mid-test → `not ok`; the `[[ ]]` as the test's last line → `not ok`. Outside bats, `bash -ec '[[ … ]]; echo survived'` printed and exited 0 while the `[ ]` form aborted — bash-3.2 errexit semantics, not a bats defect - Field measurement 2026-08-25 (`rtb-unified`, a resume-on-exception regression test): a new `expect(mock).not.toHaveBeenCalled()` was annotated as catching a "resume on any exception" widening. The fixture passed `staleQueuedJobIds: []`, so the code early-returned before the call; applying the widening left 116/116 green. The signal was in the fixture, not the assertion — an independent reviewer found it by running the mutation, and the same assertion already existed on another case, so its discriminating power for this one was zero +- Field reproduction 2026-08-30 (linkly-calendar `LinklyTabBar`, review t2-r1, fix commit 4e584cc): `totalInsetHeight` was declared as the literal 96 and a test asserted it equal to 96 while the rendered inset measured 80 pt; redefining the constant as the composed sum of its parts and comparing constant to constant turned the mismatch into an immediate failure, re-verified approve - Field reproduction 2026-08-12 (a Python health-check daemon, `heal_detector.py`): the `secret_source == "none"` branch sets `run_ok = False; send_failed = True` before the message loop, and the loop's own failure handling sets the same two. Deleting the branch entirely left all 42 tests passing, because both `none` cases supplied trigger messages that fail in the loop. Adding one case with `messages=[]` — the branch's only active writer — turned the same deletion RED in 1 test, with no assertion changed diff --git a/wiki/testing/strategy/executable-target-tests-in-swiftpm.md b/wiki/testing/strategy/executable-target-tests-in-swiftpm.md new file mode 100644 index 0000000..cf46169 --- /dev/null +++ b/wiki/testing/strategy/executable-target-tests-in-swiftpm.md @@ -0,0 +1,90 @@ +--- +id: testing-strategy-executable-target-tests-in-swiftpm +domain: testing +category: strategy +applies_to: [swift, swiftpm, xctest] +confidence: verified +sources: + - https://github.com/swiftlang/swift-package-manager/blob/main/CHANGELOG.md + - https://github.com/swiftlang/swift-evolution/blob/main/proposals/0294-package-executable-targets.md + - https://github.com/swiftlang/swift-book/blob/main/TSPL.docc/LanguageGuide/AccessControl.md + - https://github.com/swiftlang/swift-package-manager/blob/main/Sources/SPMBuildCore/BuildParameters/BuildParameters%2BTesting.swift + - https://github.com/swiftlang/swift-package-manager/issues/6367 + - https://forums.swift.org/t/executable-target-testability/52351 +last_verified: 2026-09-06 +related: [testing-strategy-test-level-choice, testing-quality-minimum-case-set] +--- + +# Unit Tests for the Internals of a SwiftPM Executable Target + +## When this applies + +A SwiftPM package has an `.executableTarget` (a macOS app or CLI built without +an Xcode project) whose internal types — layout math, formatters, parsers — +need XCTest coverage, and you are about to drop the tests or split the code +into a library target because "an executable target cannot be imported by a +test target". + +## Do this + +1. **Depend on the executable target from the test target and `@testable + import` it.** SwiftPM links the executable "as if it were a library": every + symbol except the entry point is visible to the tests. This is available to + packages whose `swift-tools-version` is `5.5` or newer. + +```swift +// Package.swift +.executableTarget(name: "DeskBat", path: "Sources/DeskBat"), +.testTarget(name: "DeskBatTests", dependencies: ["DeskBat"]), +``` + +```swift +import XCTest +@testable import DeskBat // internal types are visible +``` + +2. **Run with `swift test` and no extra flags.** `@testable import` requires + the imported module to be compiled with testing enabled; `swift test` builds + the debug configuration and SwiftPM enables testability whenever the + configuration is debug (`explicitlyEnabledTestability ?? (configuration == + .debug)`). + +| Case | Do | +|------|----| +| `swift test` on macOS or Linux, tools-version ≥ 5.5 | The two-line manifest change above; write the tests against the internal API directly | +| `swift-tools-version` below 5.5 | Raise it to `5.5` (or later) in `Package.swift` — `.executableTarget` itself needs 5.4 — then apply the row above | +| `swift test` on Windows and the executable uses a `@main` type | The link step fails with `lld-link: error: duplicate symbol: main` (swift-package-manager#6367, closed); extract the code under test into a library target for that platform's CI while keeping the direct dependency for macOS/Linux | +| The same code is consumed by two or more executables, or must ship as a `.library` product | Extract a library target and make each executable a thin wrapper — the split is justified by reuse, not by testability | +| `@testable import` fails with "module was not compiled for testing" under an Xcode scheme | Turn on the scheme's `ENABLE_TESTABILITY` for the configuration the tests build with; the executable-target dependency is not the fault | +| Running tests in release configuration | Pass testability explicitly, or drop `@testable` and test through public API — `swift test --disable-testable-imports` exists for the latter | + +3. **Keep the entry point out of the test surface.** Top-level code in + `main.swift` or the `@main` type's `main()` is the one part SwiftPM + excludes; test the functions it calls, and test the built binary itself as + a subprocess when the launch path matters ([testing-strategy-test-level-choice]). + +## Edge cases + +| Case | Then | +|------|------| +| The executable uses `main.swift` rather than a `@main` type | Same manifest; only the entry point is excluded in both forms | +| Tests must also run against the shipped binary's behavior (flags, exit codes) | Add a second test that launches the built product as a `Process` — the direct import covers the internals, the subprocess covers the launch contract | +| A pre-5.5 toolchain produced "undefined symbol" link errors for this setup | That is the pre-feature behavior the forum thread records; upgrade the toolchain and the tools-version together | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Skip unit tests for an executable target's logic | Add the executable target to `testTarget.dependencies` and `@testable import` it | Supported since tools-version 5.5 (SwiftPM CHANGELOG `#3316`); `swift test` enables testability for debug builds on its own | +| Split a library target out of the executable only to make it testable | Split only when the code has a second consumer or must be a library product | The split adds a module boundary and public API surface for no test benefit | +| Pass `-enable-testing` by hand in `unsafeFlags` | Run `swift test` as is | Debug configuration already enables testability; `unsafeFlags` blocks the package from being consumed as a dependency | + +## Sources + +- https://github.com/swiftlang/swift-package-manager/blob/main/CHANGELOG.md — Swift 5.5, `#3316`: "Test targets can now link against executable targets as if they were libraries, so that they can test any data structures or algorithms in them. All the code in the executable except for the main entry point itself is available to the unit test … This feature is available to tests defined in packages that have a tools version of `5.5` or newer"; `#4119`: `--disable-testable-imports` builds tests "without the testability feature" +- https://github.com/swiftlang/swift-evolution/blob/main/proposals/0294-package-executable-targets.md — SE-0294 `.executableTarget`, "Implemented (Swift 5.4)" +- https://github.com/swiftlang/swift-book/blob/main/TSPL.docc/LanguageGuide/AccessControl.md — "a unit test target can access any internal entity, if you mark the import declaration for a product module with the `@testable` attribute and compile that product module with testing enabled" +- https://github.com/swiftlang/swift-package-manager/blob/main/Sources/SPMBuildCore/BuildParameters/BuildParameters%2BTesting.swift — `enableTestability` resolves to `explicitlyEnabledTestability ?? (self.configuration == .debug)` +- https://github.com/swiftlang/swift-package-manager/issues/6367 — "Running `swift test` works on macOS and Linux. Running `swift test` on Windows fails" with `lld-link: error: duplicate symbol: main` +- https://forums.swift.org/t/executable-target-testability/52351 — pre-5.5 "undefined symbol" link failures for this setup; with Swift 5.5 / Xcode 13 the executable target is testable +- Field evidence 2026-08-18 (desk-bat, macOS SpriteKit app as a single `.executableTarget`): adding `.testTarget(name: "DeskBatTests", dependencies: ["DeskBat"])` and `@testable import DeskBat` made `swift test` run 49/49 green, including tests of `OverlayWindow.bottomLeftFrame` and `HistoryFormatter` inside the executable target