Found by @jaden3824 (Urusilla) in the second external conformance vector (urusilla-002, pinned urusilla @ cfb4ef3): with two operations in one assignment whose per-operation step totals are [3, 2], median_steps_per_operation reported 5.0 with denominator 1.
Root cause (lab/agentmeasure_lab/analysis.py): attempt steps were accumulated into steps_by_assignment and the median taken across assignments, while the metric name and label promise operation grain. A single assignment holding N operations therefore reports the assignment total as if it were one operation.
Fix (75027ff): steps are summed per operation key; the median runs over resolved (declared and reconciliation-passed) operations only; the denominator is the number of contributing operations. Fail-closed: unresolved operations and operations that failed #9 reconciliation contribute nothing. In the vector this yields 2.5 over 2; single-operation behavior (urusilla-001) is unchanged.
Regression guard: conformance/runners/run_external_fixture_002.py (CI) pins the per-operation totals [3, 2], the 2.5/2 median, full metrics reproduction against the externally pinned expected.json, and the fail-closed path (a tampered declaration removes that operation's steps from the median).
Same handling as #8/#9: externally found, fixed with the reporter's fixture as the regression vector. The vector is ingested at conformance/vectors/external/urusilla-002/ with the source claim boundary stated verbatim.
Found by @jaden3824 (Urusilla) in the second external conformance vector (urusilla-002, pinned urusilla @ cfb4ef3): with two operations in one assignment whose per-operation step totals are [3, 2],
median_steps_per_operationreported 5.0 with denominator 1.Root cause (
lab/agentmeasure_lab/analysis.py): attempt steps were accumulated intosteps_by_assignmentand the median taken across assignments, while the metric name and label promise operation grain. A single assignment holding N operations therefore reports the assignment total as if it were one operation.Fix (75027ff): steps are summed per operation key; the median runs over resolved (declared and reconciliation-passed) operations only; the denominator is the number of contributing operations. Fail-closed: unresolved operations and operations that failed #9 reconciliation contribute nothing. In the vector this yields 2.5 over 2; single-operation behavior (urusilla-001) is unchanged.
Regression guard:
conformance/runners/run_external_fixture_002.py(CI) pins the per-operation totals [3, 2], the 2.5/2 median, full metrics reproduction against the externally pinned expected.json, and the fail-closed path (a tampered declaration removes that operation's steps from the median).Same handling as #8/#9: externally found, fixed with the reporter's fixture as the regression vector. The vector is ingested at
conformance/vectors/external/urusilla-002/with the source claim boundary stated verbatim.