Skip to content

P1: redact-history performs ~4N git shell-outs over the notes ref (use git cat-file --batch) #31

Description

@mattfogel

Summary

redact-history shell-outs roughly 4N times over the notes ref:

  1. NotesStore::list — N+1 reads (git notes list then per-entry git notes show).
  2. Per-modified-note rewrite — N writes (git notes add --force).
  3. cache.reindex_from(&public) — re-reads N more times.
  4. cache.overlay_from(&private) — re-reads M more times.

At ~5-10ms git startup per shell-out, a 10k-note ref crosses minutes — looks hung.

Severity

P1 — UX-blocking on any non-toy notes ref. Confidence 90.

Related (P2)

The pre-push gate has a related O(N) cat-file pattern. On first push every local note is "new" → N shell-outs at 5-10ms each. Worth fixing in the same pass since the underlying primitive (git cat-file --batch) is the same. See crates/prov-cli/src/commands/hook.rs around the handle_pre_push per-blob loop.

Evidence

  • crates/prov-cli/src/commands/redact_history.rs:61-83, 106-128.
  • crates/prov-core/src/storage/notes.rs:92-119 (NotesStore::list shape).
  • crates/prov-cli/src/commands/hook.rs (pre-push N cat-file loop).

Suggested fix

Batch via git cat-file --batch for both reads (notes-show in NotesStore::list and cat-file in pre-push). For writes, the equivalent is git update-index --index-info or staging note objects with git hash-object --stdin -w + a single ref-tree rewrite. Pair with a 1k+ note benchmark in CI so a regression surfaces.

Acceptance criteria

  • Benchmark fixture: prov redact-history on a 1k-note ref completes in <2s.
  • Pre-push gate first-push case on 1k notes completes in <500ms.
  • Existing tests pass.

Reference

/tmp/compound-engineering/ce-code-review/20260502-075634-3af3d047/performance.json (perf-1, perf-2).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions