design: trail retention and rotation for hash-chained JSONL

[blitzcrieg1]

Problem

data/agentmetry-trail.jsonl is append-only, hash-chained, and unbounded. On the dogfood host it is already ~70 MB and 28,000+ lines with no documented rotation policy.

A design partner pilot will ask: what happens when this file fills the disk? Can we archive last month? Does rotation break verification?

We have partial answers in code comments but no operator-facing policy and no implementation.

What interacts

1. Hash chain (trail_chain.py, verify --trail). Truncating the file without preserving chain semantics fails verify or leaves a legacy prefix.

2. Merkle attestation (heartbeat.py, trail_merkle.py). Root computation is O(n) in trail length. Heartbeat already notes that a million-record trail should rotate or lengthen the interval.

3. External anchoring (anchor, verify --trail anchor coverage). A published root at tree_size N is a commitment the host cannot rewrite below N. Rotation must define what happens to anchor coverage for archived segments.

4. Disposition replay (disposition.py reconcile_at_boot). Boot replay assumes the trail still contains every detection_disposition event. Pruning before replay orphans triage history.

5. Evidence export / dogfood (evidence_pack.py, dogfood.py). Weekly gate reads detections and disposition state from the trail and index. Retention must not silently drop undecided findings.

6. Enterprise pilot (MSI, Intune, Splunk TA). Fleet hosts will accumulate trails independently. Pilots need a documented default before we invent per-tenant behaviour under pressure.

Questions to decide (before code)

• Segment vs truncate: New file per month/quarter with chain continuity (checkpoint sidecar) vs single file with archive copy?
• Verify scope: Does verify --trail verify one segment, all segments, or only the active tail?
• Anchor boundary: Must an anchor be taken before rotation? Is the archived segment covered only by anchors already forwarded to SIEM?
• Disposition index: Does the SQLite index become the long-lived triage record while the trail rotates, or must disposition events stay in the hot trail forever?
• Operator command: agentmetry trail rotate? Automatic at size threshold? Opt-in only for pilot?

Non-goals for the first pass

• Do not ship silent auto-delete.
• Do not break the dogfood gate or disposition replay without an explicit migration story.

Acceptance

• Written retention policy in docs (pilot-facing).
• Issue-linked implementation plan with verify/anchor/dogfood test cases.
• No code in this issue; design first.