fix(core): enumerate wisps via gc bd query, not gc bd list

[benw5483]

Summary

• wisp-compact enumerated with gc bd list --json --all -n 0 and kept select(.ephemeral == true). But bd list excludes ephemeral beads outright and doesn't even carry the ephemeral field in its projection, so that filter matched zero of 33,322 live wisps. The sweep reaped nothing, and exited 0 while it did.
• Enumeration now runs through gc bd query ephemeral=true --all --limit 0, the only listing that returns wisps, and it carries the labels the TTL policy reads.
• Classification, batching, and a per-run budget come with it, because a working enumeration surfaces a backlog the old per-bead shape could not drain inside the 300s deadline.
• Enumeration failures and empty results now exit non-zero, so a sweep that stops reaping stops looking healthy.

Why the old shape was invisible

Two things had to line up for this to stay hidden, and they did. The script swallowed enumeration trouble with || exit 0, and cmd/gc/order_dispatch.go logs an exec order's output only on a non-zero exit. So a run that saw nothing printed nothing, logged nothing, and still recorded a perfectly normal hourly execution in gc order history.

That isn't hypothetical. On the city where this turned up, the order ran hourly for two solid days without emitting a single line, while gc bd query ephemeral=true was returning 33,322 wisps the whole time, 26,619 of them closed and past TTL and therefore already due for deletion. Before that it failed loudly instead. It was 2,382 context deadline exceeded lines deep, and nobody noticed the changeover, which is rather the point: the quiet state looks exactly like success.

This is an upstream defect, not a stale materialization

Worth stating plainly. The shape of this one invites exactly the opposite conclusion, and it would be an easy call to get wrong, so here is the evidence. main carries the same select(.ephemeral == true) filter against gc bd list at wisp-compact.sh:27-28. The only difference from the older materialization found in the field is bare bd versus gc bd, which doesn't touch the filter at all, so rebuilding and reinstalling would just have swapped one silent no-op for another. I measured it against main's own enumeration, and it's the same story: gc bd list --json --all -n 0 returns 4,459 records, not one of which carries an ephemeral key, and not one of which is a wisp.

What is unchanged

Nothing here is new behavior. Retention semantics don't move at all: same TTLs, same last-label-wins resolution across the label list, same promote-versus-delete split, same comment text on promotion. Deletion stays closed-only, so a non-closed wisp past TTL still gets promoted for stuck detection rather than removed.

Throughput

The old loop spawned five jq processes and up to three date processes per bead, then one gc bd delete per bead, and each of those gc bd invocations costs roughly a second of process startup on the host I measured, no matter how little work it is actually being asked to do. That adds up fast. Now classification is a single jq pass, deletions batch through --from-file, and each run stops at a wall-clock budget. A backlog bigger than one sweep just drains across several. Promotions take the first half of the budget so a large delete backlog cannot starve stuck detection.

Three knobs, all optional: GC_WISP_COMPACT_BUDGET (default 240s), GC_WISP_COMPACT_DELETE_CHUNK (default 500), GC_WISP_COMPACT_ALLOW_EMPTY.

Known limitation, deferred out of this PR

The enumeration captures with 2>&1, so stderr from an otherwise successful query would land inside the variable that has to hold pure JSON. The run would then die in the classifier with a jq parse error, which bills the enumeration's problem to the wrong stage.

It doesn't fire today. The live query returns rc=0 with exactly zero bytes on stderr across 33,451 beads, and if it ever did fire it'd fail in the safe direction, exiting non-zero before anything is promoted or deleted. The fix is two lines and it's deliberately not here. The review rounds budgeted for this change are spent, and this is the live reclaim path that deletes on the order of 26,600 rows a run, so it's tracked separately rather than slipped in without a round covering it.

Test plan

• go test ./internal/bootstrap/packs/core/ passes, including the new TestWispCompactOrder and TestWispCompactScriptContract.

• go test ./internal/testpolicy/resourcecensus/ -run TestRepositoryLedgerMatchesCensusAndDocumentation passes. The new test adds no subprocess call site; it asserts on the embedded script's text.

• Vacuity control on the fix: the pre-change script, driven against the real gc bd list projection through a stub, makes one call, deletes nothing, prints nothing, exits 0. The reproduction of the silent no-op.

• Behavior control on the fix: the post-change script against a live 33,322-bead snapshot classifies 569 promote, 26,619 delete, 6,134 within TTL, matching an independently written classifier over the same snapshot.

• Scope control: zero of the snapshot's 627 open ephemerals appear in the delete set, and every deleted id is closed.

• Loud-fail controls, each injected separately: enumeration failure exits 1; an empty result exits 1; GC_WISP_COMPACT_ALLOW_EMPTY=1 returns that same empty case to a silent exit 0; a failing batch delete exits 1 after one log line rather than one per chunk; a zero budget does no work, reports the remainder, and exits 0.

• Vacuity control on the new test, restated. An earlier version of this bullet cited seven injected defects in a way that read as a completeness claim. It wasn't one. Eleven defects have now been injected one at a time, each reverting a single property the test claims to pin, and each turns the test red while unmodified source stays green. The count grew because review found three assertions that passed against a script whose guard had already been deleted. Two of them named a knob or a field that the operator-facing error text also carries, so deleting the guard left the name sitting there. The third pinned the loud-fail message but not the non-zero exit that message depends on, and since the controller logs an exec order's output only on a non-zero exit, flipping exit 1 to exit 0 left the test green and the sweep silent again. All three are now pinned to a construct only the guard itself can produce. Read the count as the set that's been checked, not as proof that nothing unpinned remains.

• Baseline-differenced against ./cmd/gc/. Running the TestEmbed|TestBuiltinPacks|TestCityRuntime|TestController selector on this branch leaves one failure, TestControllerStateOpenRigStoreExecProjectsRigTarget, which is the macOS /private/var versus /var family. The same selector on a pristine git archive of the base leaves six failures including that one, so the set difference of mine minus baseline is empty.

Not run: the rest of ./cmd/gc/. The diff touches one embedded shell asset plus one test file in internal/bootstrap/packs/core, and the cmd/gc tests that name wisp-compact assert the order is registered and listed, which the unchanged wisp-compact.toml still satisfies.

---

Generated by the operator's software factory.
• City: factory-main · Agent: local-core.builder-1
• On behalf of: @benw5483