fix: stream fleet-sized JSON into snapshot projections

[HelloWorldSungin]

Intent

Fix /bearings, which is completely broken in any firstmate home with a real backlog: bin/fm-bearings-snapshot.sh and bin/fm-fleet-snapshot.sh --json both exit 1 with 'jq: Argument list too long' (the same shared snapshot code also broke /fleet).

Root cause, and a correction to the original bug report: the failure is the PER-ARGUMENT limit, not the total one. Linux caps a SINGLE argv string at MAX_ARG_STRLEN (32 pages = 131072 bytes), independently of the much larger total ARG_MAX. The primary home's backlog JSON is 216172 bytes, so passing it via --argjson makes execve fail with E2BIG even though total argv size is far under ARG_MAX. The backlog note blaming ARG_MAX was wrong.

Deliberate decisions made while doing the work, so they are not mistakes:

1. Chosen fix shape: every jq payload whose size grows with the fleet is now bound with --slurpfile from a process substitution and unwrapped inside the filter by a small local 'def doc($v): ($v[0] // error(...))' helper. The reference patch in the task brief used --slurpfile plus a leading rebinding line and was explicitly offered as evidence the approach works, not as a mandate; I kept that shape because it keeps each payload's name adjacent to its value and leaves the filters' data model untouched. The 'doc' helper is deliberate: plain $x[0] would turn an empty payload into a silently-null projection, whereas --argjson used to hard-error, so 'doc' preserves the previous fail-closed behavior.

2. Scope of the audit was deliberately wider than the three sites that fail today, as the task required. Nine sites were converted: main inventory, secondmate home summary, the top-level assembly, the secondmate registry/task union, the per-secondmate record, the record accumulator, the final secondmate assembly, the secondmate landed projection, the parent-evidence reconciliation, plus Bearings' candidate-PR rows and their per-repo accumulator. The per-secondmate summary was the sharpest non-obvious one: its own byte bound FM_SNAPSHOT_SECONDMATE_MAX_BYTES defaults to 262144, twice the argv ceiling, so it could fail on its own.

3. Deliberately NOT converted: scalars, single paths, and per-record documents already bounded by the script's FM_SNAPSHOT_* read bounds (parent activities bounded at 65536 bytes / 20 records, terminal evidence at 4096, registry at 65536 / 40 records, per-task fields derived from one task's own bounded reads). These cannot grow with the fleet, and argv keeps them adjacent to the names they bind. That reasoning is recorded as a comment in fm-fleet-snapshot.sh so a future reader knows the split was considered rather than missed.

4. Explicitly out of scope by instruction: no trimming, capping, or truncating of the backlog; no restructuring of the snapshot's data model or bounds; no change to schema, field names, bounds, or TOON rendering. Output was verified byte-identical to pre-fix behavior across --json, TOON, and --secondmate-home-summary on a fixture the old code could still handle.

5. The regression test in tests/fm-bearings-snapshot.test.sh was required to actually fail without the fix, and it does ('not ok - canonical snapshot failed on an oversized backlog'). It deliberately asserts its own precondition - that the parsed backlog JSON exceeds 131072 bytes - so it fails loudly rather than silently becoming a no-op if the record shape ever shrinks. It was colocated in the existing bearings test file rather than a new runner, per repo convention.

6. AGENTS.md was deliberately left untouched. This is script mechanics, which firstmate's own coding guidelines route to the script's header/comments rather than the always-loaded contract file; adding it to AGENTS.md would violate that file's size-discipline rule.

Constraint context: this is firstmate's shared tracked material, so repo style rules apply - one sentence per line in tracked prose, plain dash never em dash, shellcheck-clean bin scripts, colocated tests. bin/fm-lint.sh and bin/fm-doc-audience-check.sh both pass. This account has pull-only access upstream, so the PR is expected to wait for upstream approval.

What Changed

• Stream fleet-sized backlog, inventory, secondmate aggregation, and candidate PR JSON into jq instead of passing it through argument strings, preventing /bearings and /fleet failures at the per-argument limit.
• Preserve fail-closed handling for empty streamed payloads without changing snapshot schemas, bounds, or rendering.
• Add regression coverage for oversized main and secondmate backlogs across fleet JSON, secondmate summaries, and Bearings output.

Risk Assessment

✅ Low: The change consistently moves fleet-growing JSON payloads off argv while preserving the existing data model, fail-closed behavior, output contract, and explicitly scoped bounds.

Testing

Diff/status baselines were clean, focused regression and adjacent-path tests passed, the base-versus-target counterfactual confirmed the argv defect, manual CLI evidence demonstrated all oversized snapshot modes, and small-fixture output remained byte-identical.

Evidence: End-to-end oversized CLI transcript

Oversized 180,063-byte backlog produced valid fleet JSON, a valid secondmate summary accounting for 200 landed records, and actual Bearings TOON with zero network calls.

OVERSIZED INPUT
parsed backlog JSON bytes: 180063
Linux single-argument ceiling used by regression: 131072

FLEET --JSON RESPONSE SUMMARY
{"schema":"fm-fleet-snapshot.v1","backlog_present":true,"backlog_records":200,"done_records":200,"main_inventory_valid":true,"secondmate_records":1,"secondmate_landed_records":1}

SECONDMATE HOME SUMMARY
{"schema":"fm-secondmate-home-summary.v1","valid":true,"landed_records":10,"total_landed":200}

BEARINGS DEFAULT TOON (actual output)
schema: fm-bearings.v1
home: fm-bearings.Oc19H0/manual-oversized
generated: "2026-07-11T18:00:00Z"
prs: "not_requested (run: /bearings include PRs)"
in_flight[3]{id,kind,state,doing}:
  external-wait,ship,paused,declared external-wait for upstream release
  scout-x,scout,done,report ready
  ship-task,ship,working,harness busy (claude-hook)
secondmates[1]{id,state,doing,provenance,freshness,age_seconds,contradiction,reason}:
  mate,captain_decision,Choose subscription order,structured-home,fresh,0,false,"-"
decisions_open[1]{id,key,verb,summary,owner}:
  mate/mate-decision-race,mate-decision-race,captain-hold,"Choose subscription order: captain choice pending",mate
landed[6]{id,what,artifact,owner}:
  landed-0200,Landed change number 0200 in the oversized fixture backlog,"https://github.com/kunchenguid/firstmate/pull/200",(main)
  mate-landed,Secondmate-managed fix,"https://github.com/kunchenguid/firstmate/pull/50",mate
  landed-0199,Landed change number 0199 in the oversized fixture backlog,"https://github.com/kunchenguid/firstmate/pull/199",(main)
  landed-0198,Landed change number 0198 in the oversized fixture backlog,"https://github.com/kunchenguid/firstmate/pull/198",(main)
  landed-0197,Landed change number 0197 in the oversized fixture backlog,"https://github.com/kunchenguid/firstmate/pull/197",(main)
  landed-0196,Landed change number 0196 in the oversized fixture backlog,"https://github.com/kunchenguid/firstmate/pull/196",(main)
gates: []
reports[1]{id,path}:
  scout-x,/tmp/fm-bearings.Oc19H0/manual-oversized/data/scout-x/report.md
recorded_prs[1]{id,url}:
  ship-task,"https://github.com/kunchenguid/firstmate/pull/9"
unhealthy_endpoints[1]{id,backend,target,exists,agent}:
  mate,tmux,"firstmate:fm-mate",true,dead
omitted[9]{surface,reveal}:
  backlog item bodies,"--fields bodies"
  task paths,"--fields paths"
  watch/steer actions,"--fields actions"
  healthy endpoint detail,"--fields endpoints"
  full scout-report inventory,"--all-reports"
  superseded queued items,"--all-queued"
  landed showing 6 of 7 (incl. 1 secondmate home(s)),"--all-landed"
  landed per-home capped at 6 for 1 home(s),"--all-landed"
  live PR discovery + checks,"--include-prs"
NETWORK CALL LOG BYTES: 0

Evidence: Base-commit counterfactual

Base commit fails on the identical oversized workload with jq: Argument list too long.

/tmp/no-mistakes-evidence/01KYXFEDG4EH9QCDSAA3QX5557/base-counterfactual/bin/fm-fleet-snapshot.sh: line 567: /home/sungin/.nix-profile/bin/jq: Argument list too long
fm-fleet-snapshot: main inventory summary failed
not ok - canonical snapshot failed on an oversized backlog

Evidence: Output compatibility evidence

Fleet JSON, secondmate summary, Bearings JSON, and Bearings TOON were byte-identical to the base implementation on a supported-size fixture.

byte-identical: fm-fleet-snapshot.sh --json
byte-identical: fm-fleet-snapshot.sh --secondmate-home-summary
byte-identical: fm-bearings-snapshot.sh --json
byte-identical: fm-bearings-snapshot.sh TOON

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

• Baseline inspection: git status --short, git diff --stat, git diff --name-only, and git diff --check a8057666..cd026797.
• Focused selector tests/fm-bearings-snapshot.test.sh::test_oversized_backlog_survives_the_per_argument_execve_limit.
• Focused selectors test_include_prs_is_the_only_fetch_path, test_pr_repository_cap_and_expansion, test_per_repository_pr_cap_is_disclosed, and test_projection_and_toon_fail_closed.
• Ran the oversized regression against base commit a805766622afb291681a15a80c5135517f1cb5ed; confirmed the expected Argument list too long failure.
• Manually exercised bin/fm-fleet-snapshot.sh --json, --secondmate-home-summary, and default bin/fm-bearings-snapshot.sh with a parsed 180,063-byte backlog.
• Used cmp to compare target and base output streams for fleet JSON, secondmate summary, Bearings JSON, and Bearings TOON on a supported-size fixture.

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.