main red after #170 merge + template hammer edits (test_apriori/test_streaming under the inline default; template pins)

[espg]

🤖 from Claude

test.yml on main has been red since the issue #170 merge chain landed, with two more failures added by the template edits that followed. Tracking issue for the fix PR.

Forensics

commit	test.yml	failures
a86fe51 (#172: force_cold flip + telemetry + self-recycle)	green	—
5f29a04 (#170 chain: compiled read path becomes the package default)	red (run 28828183316)	6
5c9b1e2 ("cold restart hammer", template-only)	red (run 28828436029)	8
260221b (RecycleMaxInvocations default "0"→"1")	red	8 (:212 now '1' != '8')

Full failure list (reproduced locally on 260221b):

From the #170 default flip (6):

• tests/test_apriori.py::TestWorkerSeam::test_flag_off_call_signature_unchanged — 'No data after filtering (1 group reads raised)' != 'No data after filtering'
• tests/test_streaming.py::TestStreamingWorker::test_single_buffer_is_byte_identical_to_pooled — KeyError: 'h_tdigest'
• tests/test_streaming.py::TestStreamingWorker::test_multi_flush_counts_exact_quantiles_close — KeyError: 'count'
• tests/test_streaming.py::TestStreamingWorker::test_streaming_releases_buffer_per_flush — assert 0 > 0
• tests/test_streaming.py::TestStreamingWorker::test_streaming_empty_shard_matches_pooled_no_data — error-message mismatch as above
• tests/test_streaming.py::TestStreamingReviewFolds::test_profile_charges_merge_to_read_phase — {'read'} != {'aggregate', 'index', 'read'}

From the template edits (2):

• tests/test_lambda_build.py::TestTemplateEnvironment::test_self_recycle_env_defaults — RecycleMaxInvocations default pin (now '1' != '8')
• tests/test_lambda_build.py::TestTemplateEnvironment::test_execution_role_grants_zagg_index_store — the template's zagg-index role statement was broadened from Get/Put on arn:aws:s3:::sliderule-public-cors/zagg-index/* to Get/Put/Delete on arn:aws:s3:::sliderule-public-cors/*

Verdict on the #170 group: test artifact, not a regression

All six failures share one mechanism. The tests drive process_shard with doubles aimed at the hierarchical seam: monkeypatch.setattr("zagg.processing._read_group", fake) plus H5Coro = lambda *a, **k: object(). That seam is only wired through HierarchicalIndex.read_group, which late-binds _processing._read_group specifically so monkeypatched fakes keep working (its own docstring, src/zagg/index/hierarchical.py). InlineIndex.read_group (src/zagg/index/inline.py) never calls zagg.processing._read_group — it routes through _planned_read_group/_read_group_full with a compiled read_fn that tries build_chunk_map(h5obj, …) and falls back to h5obj.readDatasets(…). Against the fake object() that raises ('object' object has no attribute 'readDatasets'), the worker's per-group handler counts it as a read error, and the issue #116 accounting correctly reports "No data after filtering (N group reads raised)".

The plain vs. "(N group reads raised)" message split predates #170 (issue #116, worker.py) and is unchanged; real callers with real HDF5 objects see identical semantics on both backends. What changed is only which backend the unpinned test configs exercise by default. Note test_apriori.py::TestWorkerSeam::test_granule_url_passed_when_enabled still passes — its chunk_boundaries source keeps the hierarchical default per the #170 carve-out in index_from_config.

Fix: pin index: {backend: hierarchical} in the test configs that test the _read_group seam / worker mechanics through monkeypatched fakes — the same approach PR #173 took for the benchmark-config pins. No production code change.

Template fixes

• 260221b makes default RecycleMaxInvocations="1" (recycle after every invocation = cold-every-time) the deliberate posture; the fix updates test_lambda_build.py:212 to pin "1" rather than reverting the template.
• The zagg-index role statement is restored to the issue Virtual index entry point: pluggable chunk-index backends for the read path #160 scope (Get/Put on zagg-index/* only): the comment directly above it still reads "scoped to the public zagg-index prefix only", the admin copy deployment/aws/execution_role.yaml still carries the scoped form (the two are pinned to lockstep by the same test), and nothing in-tree issues DeleteObject. If the broader grant is actually needed for the cold-restart hammer work, that wants a deliberate commit + test update — question raised on the fix PR.

[espg]

🤖 from Claude

Scope additions on the fix PR (#176), per espg direction there: (1) the role grant is Get/Put/Delete on the whole sliderule-public-cors bucket (deliberate — index write-back and worker-written output zarr stores), not the narrow #160 restore this issue originally proposed; (2) phase 4 materializes the CloudWatch error-signal split (zagg/lambda metric filters: per-function SelfRecycleCount / WorkerErrorCount) since the RecycleMaxInvocations=1 posture makes the raw Lambda Errors metric 100% noise. Known follow-up gap (not this PR): streaming/worker integration coverage of the compiled inline read path needs real HDF5 fixtures — the suite currently exercises it only via unit tests in test_index.py.