Generic-tier emission strips agent_id_hash + agent_name + audit_signature on duplicated traces

[emooreatx]

Context

First post-2.7.9 export to RATCHET (per #4) shows that 14 of 89 traces are dual-emitted at both detailed and generic tiers — same trace_id, same signing_key_id, but the generic copies systematically drop agent identity. Spec §3.1 says this shouldn't happen.

Export: release_v2/data/lens_export_v2_fixed_2026-05-03T04-00-04Z/
Rows examined: 1,901 trace_events

Finding

trace_level	rows	agents	agent_id_hash	agent_name	ACTION_RESULT.audit_signature
detailed	1,636	9 distinct (proper hashes)	populated	populated	89/89 populated
generic	265	1 ("unknown")	"unknown" on 100%	null on 100%	0/14 populated (entry_hash + sequence_number ARE populated)

Critical observation: the 265 generic-tier rows are signed by signing_key_id="agent-ee2271f272d2" — the same key as detailed-tier agent ee2271f272d21fff. Spec §3.1 says agent_id_hash MUST be sha256(public_key_bytes)[..16] of the signing key. Same key implies same hash. Reality: the same key produces agent_id_hash="ee2271f272d21fff" on 309 detailed-tier rows and agent_id_hash="unknown" on 265 generic-tier rows.

Pattern is byte-equivalent in structure: the 14 dual-emitted traces each have 21-row detailed copies and 21-row generic copies with identical event_type distributions (THOUGHT_START, SNAPSHOT_AND_CONTEXT, LLM_CALL×N, DMA_RESULTS, IDMA_RESULT, ASPDMA_RESULT×N, CONSCIENCE_RESULT, ACTION_RESULT).

Spec text being violated

CIRISAgent/FSD/TRACE_WIRE_FORMAT.md @ v2.7.9-stable §3.1:

agent_id_hash appears at BOTH the CompleteTrace envelope level (§3) AND on every TraceComponent (§4). The two values MUST be equal — agents emit them locked at build time; persistence MAY reject mismatches as malformed.

§3, agent_id_hash definition:

First 16 hex characters (8 bytes) of sha256(public_key_bytes), lowercase. Canonical derivation: hashlib.sha256(public_key_bytes).hexdigest()[:16] over the agent's Ed25519 signing key bytes. ... implementations MUST use exactly this derivation so the values are interchangeable across consumers.

"unknown" is not a valid output of sha256(public_key_bytes).hexdigest()[:16]. The generic-tier path is bypassing the canonical derivation.

Audit-signature gap is a corollary

Of 103 ACTION_RESULT rows, 89 have full audit anchor (audit_entry_hash, audit_sequence_number, audit_signature all populated) and 14 have a partial anchor (entry_hash + sequence_number populated, signature null). The 14 partial-anchor rows are exactly the 14 generic-tier ACTION_RESULT rows. So generic-tier emission also drops the audit-chain signature, leaving the chain entries unverifiable.

Same root cause: whatever the generic-tier emission path is doing, it's not running the same identity-derivation + signing logic as the detailed path.

Possible root causes

1. Privacy-by-tier intent (undocumented): generic tier deliberately strips agent identity for privacy. If so, spec §3.1 needs an exception clause. But this contradicts the "MUST be equal" clause and breaks N_eff measurement at generic tier.
2. Generic-tier emission path bug: the lens-side or persist-side path that emits the generic-tier copy doesn't propagate the agent's agent_id_hash from the envelope, defaulting to "unknown". Same path doesn't sign the audit anchor.
3. Schema-version mismatch: the generic-tier copy may have been emitted at schema_version="2.7.0" (envelope-only agent_id_hash) and the export query coerced it to "2.7.9". But export filter is schema_version='2.7.9', so this shouldn't slip through.

Why this matters for RATCHET

• 265 of 1,901 rows (13.9%) are unusable for per-agent or N_eff analysis — they have no resolvable agent identity.
• 14 of 89 traces (16%) have weakened audit-chain integrity at the seal event.
• The 14 dual-emitted traces represent the same thoughts as the detailed-tier copies; deduplicating them at analysis time leaves only the detailed copy. Functional workaround for RATCHET, but doubles the storage cost in the corpus.

What this repo needs to do

1. Document the intent: is generic-tier supposed to strip agent identity? If yes, this is a spec issue (§3.1 needs exception clause + audit-anchor handling for generic). If no, this is a code bug.
2. If bug: trace the generic-tier emission path — likely on the lens or persist side, since the agent's signing key is the same one used for detailed-tier emission.
3. If intent: open a CIRISAgent spec PR to add the generic-tier exception, and clarify how N_eff measurement is supposed to work without per-agent attribution at that tier.

Non-blocking for RATCHET

Detailed-tier subset (1,636 rows, 9 agents, 75 detailed-only traces + 14 dual-emitted) is sufficient for Layer 1/2 federation analysis. Filing so the convergence happens before quarterly cadence settles.

References

• Spec being violated: CIRISAgent/FSD/TRACE_WIRE_FORMAT.md @ v2.7.9-stable §3 + §3.1
• Spec lock-in commit: CIRISAgent 4c28910d8 ("denormalize agent_id_hash onto every TraceComponent for 2.7.9")
• Privacy boundary spec: CIRISLens/docs/THREAT_MODEL.md §1
• Persist threat model AV-9 (cross-agent dedup-key collision): closure depends on agent_id_hash being correctly populated; "unknown" agent_id_hash creates a single-tenant collision space at generic tier
• Adjacent: Post-2.7.9 QA-traffic export pipeline for RATCHET validation corpus #4 (RATCHET export pipeline)

[emooreatx]

Investigation: pattern is wider than the corpus, definitively agent-side

Production scan beyond the corpus filter:

SELECT trace_level, schema_version,
       COUNT(*) FILTER (WHERE agent_id_hash = 'unknown') AS unknown_hash,
       COUNT(*) FILTER (WHERE agent_id_hash != 'unknown' AND agent_id_hash IS NOT NULL) AS real_hash
FROM cirislens.trace_events
WHERE ts > NOW() - INTERVAL '2 days'
GROUP BY 1, 2;

 trace_level | schema_version | unknown_hash | real_hash
-------------+----------------+--------------+----------
 detailed    | 2.7.0          |            0 |     2371   ← always canonical at detailed
 detailed    | 2.7.9          |            0 |     1647
 generic     | 2.7.0          |         2390 |     2419
 generic     | 2.7.9          |          673 |     3024

Detailed-tier emissions never produce "unknown". Generic-tier emissions produce both — even within a single agent. Single-agent example: agent-ee2271f272d2 at 2.7.9 ships 717 rows with the canonical ee2271f272d21fff AND 673 rows with unknown in the same time window. Same key, two paths, divergent identity treatment.

That same-agent-both-values pattern rules out the "privacy-by-tier intent" hypothesis from the original issue body — privacy intent would be all-or-nothing per agent. It's a code-path divergence in the agent's emission stack: detailed-tier always runs the canonical-derivation step; generic-tier sometimes does, sometimes doesn't.

Filed upstream

• CIRISAgent#716 — agent-side fix for the divergent generic-tier emission path. Includes the audit_signature null on ACTION_RESULT corollary (same root cause: shorter generic-tier emission stack skips identity-derivation AND audit-chain signing).

Lens-side action

None. Whatever the generic-tier code path is doing, it's in the agent's emission stack; lens just receives + persists what arrives. Detailed-tier subset is correct on every measure.

Closing this lens-side once the agent fix lands + new corpus export shows canonical hashes on generic-tier rows.