From 8480d010a22bf798cc40bfa23b121a2aa9b5cb8a Mon Sep 17 00:00:00 2001 From: Matt Fogel Date: Thu, 14 May 2026 08:43:26 -0400 Subject: [PATCH 1/2] skill: rewrite prov skill to be question-triggered, not edit-triggered MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous shape asked Claude Code to call `prov log` before any non-trivial edit. That created context bloat: the skill description matched broad "refactor / debug / extend" intent, so the skill loaded into context for almost every substantive code change, and the agent spent tool calls on `prov log` lookups the user never asked for. This rewrites the skill so it activates only when the user asks an origin / intent / history question — "why does this do X", "what was the prompt that wrote this", "what's the history of this file", "find the prompts where we decided Y", "is this drifted". The description, the trigger reference, and the smoke-test plan all reinforce the same boundary: edit / refactor / debug asks should NOT preemptively run `prov log`; only explicit provenance questions should. Capture hooks are unchanged — they write to staging only and don't inject anything into the agent's prompt, so they don't cause the bloat this rewrite targets. --- README.md | 2 +- plugin/.claude-plugin/plugin.json | 2 +- plugin/README.md | 6 +- plugin/skills/prov/SKILL.md | 218 ++++++++++----------- plugin/skills/prov/references/querying.md | 154 ++++++++------- plugin/skills/prov/references/triggers.md | 219 +++++++++++++--------- plugin/skills/prov/tests/skill_smoke.md | 166 ++++++++++------ 7 files changed, 437 insertions(+), 330 deletions(-) diff --git a/README.md b/README.md index b8c5af8..310c3e2 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Prov captures the prompt-and-conversation context behind AI-agent-driven edits, attaches it to commits via git notes, and exposes it through thin read surfaces: - **CLI** for humans — `prov log src/auth.ts:42` returns the originating prompt for any line. -- **Agent skills and hooks** for supported harnesses — Claude Code and Codex can capture provenance today, and agents can query their prior reasoning before proposing edits. +- **Agent skills and hooks** for supported harnesses — Claude Code and Codex capture provenance automatically during sessions, and an agent skill teaches the agent to answer the user's provenance questions ("why does this do X", "what was the prompt that wrote this", "what's the history of this file") on demand. ## How it works diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 21c9ea6..9ec036d 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "prov", - "description": "Prompt provenance for Claude Code: capture each turn's prompt and edits as a git note on the resulting commit, then surface the originating prompt for any line via `prov log :`. Bundles the capture hooks (UserPromptSubmit, PostToolUse, Stop, SessionStart) and a Skill that teaches the agent to query its own prior reasoning before substantive edits.", + "description": "Prompt provenance for Claude Code: capture each turn's prompt and edits as a git note on the resulting commit, then surface the originating prompt for any line via `prov log :`. Bundles the capture hooks (UserPromptSubmit, PostToolUse, Stop, SessionStart) and a Skill that teaches the agent to answer user questions about code provenance — origin, intent, history, drift — using `prov log` and `prov search`.", "version": "0.1.1", "author": { "name": "Matt Fogel", diff --git a/plugin/README.md b/plugin/README.md index 40fdd6a..c601091 100644 --- a/plugin/README.md +++ b/plugin/README.md @@ -7,8 +7,10 @@ for [prov](https://github.com/mattfogel/prov), the prompt-provenance tool. It bu (`UserPromptSubmit`, `PostToolUse` matched on `Edit|Write|MultiEdit`, `Stop`, `SessionStart`). Each hook calls `prov hook ` with a 5-second timeout. - `skills/prov/SKILL.md` — teaches the agent to query its own prior reasoning - (`prov log :`) before substantive edits. *(See U12 in the plan; - the SKILL ships in a separate unit and is referenced here for completeness.)* + (`prov log :`, `prov search `) when the user asks about + code provenance ("why does this do X", "what was the prompt for this line", + "what's the history of this file"). The skill is question-triggered, not + edit-triggered — it does not preemptively query before refactors or edits. ## Prerequisites diff --git a/plugin/skills/prov/SKILL.md b/plugin/skills/prov/SKILL.md index ce8903d..b9de07e 100644 --- a/plugin/skills/prov/SKILL.md +++ b/plugin/skills/prov/SKILL.md @@ -1,136 +1,136 @@ --- name: prov -description: Query prompt provenance for AI-written code before refactoring, rewriting, or editing it. Use this skill when a user asks to refactor, modify, debug, or extend a non-trivial section of an existing file that may have been written by a supported agent harness in a prior session — `prov log :` returns the originating prompt, model, conversation, harness, and drift state for any line, so the agent can plan in light of the constraints the original turn was written against. Also use to recover the original intent for AI-written code whose surrounding context has changed, to detect when a line has been hand-edited away from its original AI capture, and to read the prompt history of a file before making structural changes. Skip for greenfield writes, single-line trivial fixes, formatting, lint, and pure documentation/config edits. -paths: - - "**/*" - - "!**/*.md" - - "!**/README*" - - "!**/CHANGELOG*" - - "!**/*.json" - - "!**/*.yaml" - - "!**/*.yml" - - "!**/*.toml" +description: Use when the user asks about code provenance — "why does this code do X", "what was the original prompt for this line", "who or what wrote this", "what's the history of this file", "find the prompts where we decided Y", "has this been rewritten and what came before", "was this AI-written, by which model, in which session". Calls `prov log [:]` to surface the originating prompt, model, conversation, and drift state, and `prov search ` to find prompts by content. Use whenever the user wants the *why* behind code that's already on disk, the prompt history of a file, or the conversation a line came out of. Do NOT auto-run before edits, refactors, or debugging — only when the user has actually asked an origin/history/intent question. license: MIT --- -# prov — query your own prior reasoning before substantive edits - -This skill teaches the agent to ask "what was the prompt that wrote this code?" -before refactoring, rewriting, or extending an existing file. It exists because -AI-written code carries constraints that aren't visible in the diff: a prompt -like *"keep the dedupe window at 90 days for compliance"* shapes the resulting -code in ways that will look like arbitrary defaults to a future agent reading -only the result. - -## What this skill does - -Calls `prov log : --only-if-substantial --json` (point lookup) or -`prov log --only-if-substantial --json` (whole-file context) to -retrieve the originating prompt for AI-written code. Surfaces the most -relevant prior turn into the planning step so the new edit treats the past -constraint as load-bearing unless the current request explicitly invalidates -it. - -The `--only-if-substantial` flag returns empty for files under 10 lines or -with no existing notes — this is the CLI-level gate that keeps the skill -quiet on greenfield code and trivial files. - -## When to use it - -Query provenance **before proposing edits** in any of these situations: - -- The user asks for a substantive change (refactor, rewrite, extract, inline, - rename across multiple call sites, redesign) to a file that already exists - and has multiple non-trivial blocks. -- The user asks to debug behavior whose original constraints aren't obvious - from the code alone — defaults that look arbitrary, magic numbers, defensive - branches, retry counts, dedupe windows, validation rules. -- The user asks to extend AI-written code in a way that depends on assumptions - the original author made (e.g., adding a new error path that has to compose - with the original error strategy). -- The user asks "why does this code do X" or "what was the original intent" — - the prompt is the answer. -- A line has visibly drifted from its original AI capture (commit history - shows hand edits since) — provenance reveals the original intent and the - drift state, so the new edit can decide whether to preserve or supersede. - -## When NOT to use it - -Skip provenance queries for: - -- Greenfield writes — creating a new file or appending purely additive code - to an existing one. -- Single-line trivial changes — typo fixes, comment edits, formatting, lint - fixes, import sorting. -- Pure documentation or config edits — the `paths:` glob already excludes - `*.md`, `README*`, `CHANGELOG*`, `*.json`, `*.yaml`, `*.yml`, `*.toml`, but - if the user asks about a config-adjacent file inside excluded scope, don't - override the exclusion. -- Files the skill returns empty for — `--only-if-substantial` already - filtered them out; an empty response means "no provenance, proceed - normally." - -If unsure whether the change is substantive, see `references/triggers.md`. +# prov — answer user questions about code provenance + +`prov` is a local tool that records the prompt-and-conversation context behind +AI-authored edits and attaches it to commits via git notes. This skill is how +you turn that buried context into answers when the user asks about it. + +## What prov is + +- **Capture happens automatically** in the background: harness hooks stage the + prompt + edits during a session, and a `post-commit` git hook attaches the + staged context to the resulting commit as a note on `refs/notes/prov` (or + `refs/notes/prov-private`). +- **You read it via the CLI.** The two surfaces you'll use are `prov log` and + `prov search`. Output is human-readable by default; pass `--json` when you + want a machine-readable envelope. +- **Notes are local-only by default.** If `prov log` returns no provenance, + the file is either older than the install, was authored by a human, or + comes from a teammate who hasn't pushed notes. + +## When to use this skill + +Activate when the user asks for the *origin*, *intent*, or *history* of code +that already exists. Concrete user phrasings that should trigger you: + +- "Why does `X` do this?" / "Why is `Y` set to `Z` here?" +- "What was the prompt that wrote this line / function / file?" +- "Did I write this, or did the agent?" / "Which model wrote this?" +- "What's the history of this file?" / "Show me the prompts behind + `src/foo.rs`." +- "Has this been rewritten? What did it look like before?" +- "Find the prompts where we talked about rate limiting / dedupe / retries." +- "What session was this from?" / "What conversation produced this code?" +- "Has someone hand-edited this since the AI wrote it?" (drift question) + +If the user is asking you to *change* code — refactor, debug, extend, +rewrite, fix — **do not** preemptively run `prov log`. Only query if the +user themselves asks an origin/intent question. + +For the full mapping of user phrasings to queries, see +`references/triggers.md`. ## How to query -Two patterns. See `references/querying.md` for full examples and JSON shapes. +Three patterns cover almost every question. Full example output and JSON +shapes in `references/querying.md`. + +**Point lookup — "who/what wrote this specific line"** + +```bash +prov log src/payments.ts:247 +``` + +Returns the originating prompt, model, conversation id, turn index, and +drift status (`unchanged` | `drifted` | `no_provenance`) for that line. -**Point lookup** — when the user asks about a specific line or a small -contiguous range: +**Whole-file history — "what prompts have shaped this file"** ```bash -prov log src/payments.ts:247 --only-if-substantial --json +prov log src/payments.ts ``` -Returns `{ status: "unchanged" | "drifted" | "no_provenance", prompt, model, -conversation_id, turn_index, blame_commit, ... }` for that line. +Returns every captured edit on the file, most recent first. -**Whole-file context** — when the change spans the file or you want the -file's prompt history before planning: +**Cross-file search — "where did we decide X"** ```bash -prov log src/payments.ts --only-if-substantial --json +prov search "rate limiting" ``` -Returns `{ edits: [{ line_start, line_end, prompt, model, timestamp, ... }], -history: [...] }`. An empty `edits` array means no provenance for this file -— proceed without it. +Full-text search across captured prompts. Returns matching prompts with the +files and commits they touched. + +Add `--json` when you want to parse the response programmatically (e.g., to +pick a single field or correlate with other tool output). Add `--history` +to `prov log` to walk the `derived_from` chain and surface superseded prior +prompts when an AI rewrite replaced an earlier AI edit. + +## How to answer with the result -## How to use the result +Cite the prompt **verbatim**. The exact wording is the provenance — the +original phrasing carries constraints that paraphrase will lose. -When provenance is found: +A good answer for an unchanged line: -1. **Cite the prompt verbatim** in the planning step before proposing the - edit. Don't paraphrase. The exact wording carries the constraint. -2. **Identify the load-bearing constraint** — what did the original prompt - ask for that shaped the current code? "Keep dedupe at 90 days", "validate - email RFC 5322 strictly", "fail closed on rate limit" — these are - constraints, not arbitrary choices. -3. **Decide whether the current request invalidates that constraint.** If - the user explicitly asks to change it ("change the dedupe window to 30 - days"), the constraint is now stale — proceed. If the user asked for an - adjacent change ("add retry logic"), preserve the constraint. -4. **If the line is `drifted`**, the current code no longer matches the - original AI capture — a human has edited it since. Frame the explanation - around both the original intent and the divergence. Be careful: a drifted - line often encodes a deliberate human override of AI logic, and rewriting - it back to match the prompt may regress whatever bug the human was fixing. +> Line 247 came from turn 4 of session `sess_abc123` on 2026-03-12, written +> by `claude-sonnet-4-5` against this prompt: +> +> > "Add a 90-day dedupe window on payment intents — compliance requires we +> > never charge twice within a quarter even if the idempotency key +> > collides." +> +> So the 90-day window is a deliberate compliance constraint, not an +> arbitrary default. -When provenance is empty: +A good answer for a drifted line — surface both the original intent and the +divergence: -- Proceed normally. Empty isn't an error — it just means the file is short, - has no notes, or is excluded by `--only-if-substantial`. +> `prov log` reports this line as drifted. The original AI capture (prompt: +> "add a 90-day dedupe window for compliance") asked for 90 days; the +> current value is 30. `blame_author_after` shows `alice@example.com` +> changed it on 2026-04-02 in commit `b3c4d5e`. That looks like a deliberate +> human override — worth checking with Alice before assuming the 30-day +> value is wrong. + +A good answer when there's no provenance: + +> No provenance recorded for that line. Likely human-authored, or it +> predates `prov install` in this repo. + +## What to do with the result besides answering + +The user may use your answer to inform their next ask. If they follow up +with "OK, then change it to 60 days", you now know the constraint was +load-bearing — flag the compliance angle before making the change. But +running `prov log` is only justified once the user has asked an +origin/intent question, not as a preemptive check. ## Failure modes -- `prov` not on PATH: the binary install hasn't run. Skip the query and - proceed normally; surface a one-line note to the user that `prov log` - is unavailable. -- Repo has no `.git/prov.db`: the user hasn't run `prov install` here. Skip - and proceed. -- `prov log` returns `status: "no_provenance"`: the line has no resolvable - note (could be human-authored, could be a fresh repo). Proceed without it. +All of these mean "skip the query and tell the user plainly": + +- `prov: command not found` — the binary isn't installed. Say so. +- `not in a git repo` — the cwd isn't inside a git repo. +- `.git/prov.db` missing — `prov install` hasn't been run here. +- `status: "no_provenance"` (point lookup) — the line has no resolvable + note. Could be human-authored, could be from before the install. +- Empty `edits` array (whole-file lookup) — no captured edits on this file. -Hooks fail non-blocking by design; absence of provenance is never a reason -to refuse the user's request. +None of these are errors. They just mean the answer to the user's question +is "we don't have provenance for that," which is a perfectly fine answer. diff --git a/plugin/skills/prov/references/querying.md b/plugin/skills/prov/references/querying.md index b046b29..0709c25 100644 --- a/plugin/skills/prov/references/querying.md +++ b/plugin/skills/prov/references/querying.md @@ -1,21 +1,22 @@ # Querying provenance -Concrete patterns for calling `prov log` from the agent loop, with example -JSON output and how to integrate findings into the planning step. +Concrete patterns for calling `prov log` and `prov search` when the user has +asked about the origin, intent, or history of code. Example JSON shapes and +notes on how to compose the answer. -## Two query shapes +## Three query shapes -### Point lookup — `prov log : --json` +### Point lookup — `prov log : [--json]` -Use when the user references a specific line, or when the proposed change -touches a small contiguous range and you want the originating prompt for -that range. +Use when the user asks about a specific line or a small contiguous range +("why is line 247 doing X", "what was the prompt for this if-branch"). ```bash -prov log src/payments.ts:247 --only-if-substantial --json +prov log src/payments.ts:247 --json ``` -Example output for an unchanged line: +Example output for an **unchanged** line (current code still matches the +original AI capture): ```json { @@ -32,7 +33,8 @@ Example output for an unchanged line: } ``` -Example output for a drifted line (current code no longer matches AI capture): +Example output for a **drifted** line (current code differs from the +original AI capture — someone hand-edited it): ```json { @@ -50,7 +52,7 @@ Example output for a drifted line (current code no longer matches AI capture): } ``` -Example output when no provenance is available: +Example output when there is no provenance for the line: ```json { @@ -62,13 +64,14 @@ Example output when no provenance is available: } ``` -### Whole-file context — `prov log --json` +### Whole-file history — `prov log [--json]` -Use when the change spans the file or you want the prompt history before -planning a structural edit. +Use when the user asks about the file overall ("what's the history of +`src/payments.ts`", "what prompts have shaped this file", "show me every AI +edit on this file"). ```bash -prov log src/payments.ts --only-if-substantial --json +prov log src/payments.ts --json ``` Example output: @@ -101,68 +104,89 @@ Example output: } ``` -The `edits` array is ordered by recency (most recent first). `history` -carries superseded prior prompts when an AI rewrite replaced an earlier AI -edit on the same span. +The `edits` array is ordered most-recent first. `history` is populated when +you pass `--history`: it carries superseded prior prompts that an AI rewrite +replaced. -## Empty result handling +### Prompt search — `prov search [--json]` -`--only-if-substantial` returns empty for short files (< 10 lines) or files -with no existing notes: +Use when the user asks "where did we decide X" or "find the prompts where +we talked about Y" — the question isn't about a specific file or line, it's +about a topic that may have surfaced in multiple prompts. -```json -{ "file": "src/utils.ts", "edits": [], "history": [], "prov_version": "0.1.1" } +```bash +prov search "rate limiting" --json ``` -Empty is not an error. Proceed with the edit normally. - -## Integrating into planning - -When the result has provenance, surface the prompt into your planning step -**before** proposing code changes. Example planning frame: - -> Before refactoring `src/payments.ts:247`, I checked the originating prompt -> via `prov log`. The line was written in turn 4 of session `sess_abc123` on -> 2026-03-12 against the prompt: -> -> > "Add a 90-day dedupe window on payment intents — compliance requires we -> > never charge twice within a quarter even if the idempotency key collides." -> -> The 90-day window is therefore a load-bearing constraint, not an arbitrary -> default. The user's current request ("rename the field to `windowDays`") -> is a cosmetic change and should preserve the value. I'll rename without -> changing the constant. - -If the line is `drifted`, frame both the original intent AND the divergence: - -> `prov log` reports this line as drifted — the current code differs from -> the original AI capture (the prompt asked for 90 days; the current value -> is 30). `blame_author_after` shows alice@example.com edited it on -> 2026-04-02. Likely a deliberate human override; before changing it again -> I'll ask whether the 30-day value is intentional. +Returns matching prompts with the commits and files they touched. Useful +for tracing the lineage of a decision across a codebase ("when did we first +introduce rate limiting? which prompts framed it?"). + +## Useful flags + +- `--json` — machine-readable envelope. Use whenever you'll pick a single + field or correlate with other tool output. The human-readable rendering + is fine when you'll just quote the prompt back to the user. +- `--history` — on `prov log`, walks `derived_from` so AI-on-AI rewrites + show the superseded prior prompts. Useful for "what did this look like + before". +- `--only-if-substantial` — returns empty for files under 10 lines or files + with no captured notes. Rarely needed here: the user has *already* asked + about a specific file, so substantiality is implicit. Use it only if you + want to suppress noise on tiny files. +- `--full` — reserved for future transcript expansion; currently prints the + stored `preceding_turns_summary`. + +## Composing the answer + +When provenance is present, **quote the prompt verbatim** — paraphrase +loses constraint nuance ("90-day dedupe window" carries the unit and the +implied compliance horizon; "a long dedupe window" doesn't). + +Include the load-bearing metadata: model, conversation id, turn index, +timestamp, commit. The user often cares about *which session* introduced +the code so they can find the conversation. + +For drifted lines, name both the original intent and the divergence so the +user can decide which to trust: + +> Originally written by `claude-sonnet-4-5` against the prompt "add a 90-day +> dedupe window for compliance"; the current value is 30 days, edited by +> `alice@example.com` on 2026-04-02 in commit `b3c4d5e`. Two plausible +> reads: Alice deliberately overrode the compliance default, or this was an +> ad-hoc tweak that broke the original constraint. Worth confirming with +> Alice. + +For `no_provenance` or empty results, say so plainly. Don't invent +explanations — the line is either human-authored, predates `prov install`, +or comes from a teammate who hasn't shared their notes ref. ## Bash idioms -If you want a single field from the JSON output, pipe through `jq`: +Pick a single field from JSON output with `jq`: ```bash -prov log src/payments.ts:247 --only-if-substantial --json | jq -r '.prompt' +prov log src/payments.ts:247 --json | jq -r '.prompt' +prov log src/payments.ts --json | jq -r '.edits[] | "\(.timestamp) \(.prompt)"' ``` -For shell-based agents that don't have `jq` available, parse the JSON -directly in your tool runtime. The shape is stable across `prov_version` -within v0.x; consult `prov_version` in the envelope before trusting future -fields not documented here. +If `jq` isn't available, parse the JSON in your tool runtime. The shape is +stable across `prov_version` within v0.x; consult the `prov_version` field +before relying on fields not documented here. ## Failure modes to handle gracefully -- `prov: command not found` — the binary isn't on PATH. Skip and proceed. -- `not in a git repo` — the cwd isn't inside a git repo. Skip. -- `.git/prov.db` missing — user hasn't run `prov install` here. Skip. -- Empty stdout (with `--only-if-substantial`) — file is short or has no - notes. Proceed without provenance. -- `status: "no_provenance"` on a point lookup — the specific line has no - resolvable note. Proceed without it. - -In every failure case, the right move is to proceed with the edit. Provenance -is additive context; its absence never blocks the user's request. +- `prov: command not found` — the binary isn't on PATH. Tell the user + `prov` isn't installed and offer the install command from the project + README. +- `not in a git repo` — answer "I can't query provenance outside a git + repo." +- `.git/prov.db` missing — `prov install` hasn't been run here. Tell the + user. +- Empty stdout / `edits: []` — no captured edits. Tell the user plainly. +- `status: "no_provenance"` on a point lookup — the line has no note. + Likely human-authored or predates install. + +In every failure case the right move is to surface the absence to the user +and answer their question with what you can derive from other sources +(`git blame`, `git log`, the code itself). diff --git a/plugin/skills/prov/references/triggers.md b/plugin/skills/prov/references/triggers.md index bccb076..3ece810 100644 --- a/plugin/skills/prov/references/triggers.md +++ b/plugin/skills/prov/references/triggers.md @@ -1,93 +1,126 @@ -# Triggers — when is the change substantive enough to query? - -Heuristics for deciding whether to call `prov log` before proposing an edit. -Use this when the user's ask sits in the gray zone between "trivial" and -"substantive." - -## Default rule of thumb - -Query provenance if **all three** are true: - -1. The file already exists with non-trivial content (more than ~10 lines). -2. The change touches behavior, structure, or public surface — not pure - cosmetics. -3. The user's request would change how the code *works*, not just how it - *looks*. - -If any of the three is false, skip the query. - -## Substantive — query first - -These changes warrant a `prov log` call before proposing code: - -- **Refactor** — extract function, inline, rename across call sites, - reorganize control flow, change a class hierarchy. -- **Rewrite** — replace a block with a different implementation of the same - behavior. -- **Behavior change** — change a default, threshold, retry policy, validation - rule, error strategy, timeout, cache key, or any value/condition that - alters runtime behavior. -- **Public surface change** — modify a function signature, add/remove a - parameter, change an exported type, alter an API contract. -- **Debugging an existing bug** — when the user asks "why does this happen" - and the answer might be encoded in the original prompt's constraints - (e.g., a deliberate trade-off the prompt called for). -- **Adding error handling around AI-written code** — the original error - strategy may already exist; a new layer can conflict with retry middleware, - fallback logic, or framework error handling. -- **Adding a new branch to existing logic** — branches compose with the - control flow the prompt shaped; the prompt may have explicitly excluded - the branch you're adding. - -## Trivial — skip the query - -These changes do not warrant a query: - -- **Typo fixes** — single-character or single-word corrections. -- **Comment edits** — adding, removing, or rewording comments without - touching code. -- **Formatting** — whitespace, indentation, line wrapping, import sorting. -- **Lint fixes** — applying a linter rule (unused import removal, prefer- - const, etc.) where the rule itself dictates the change. -- **Pure rename of a single local variable** — when the rename doesn't - cross file boundaries and doesn't change the public surface. -- **Adding a log line** — `console.log`, `tracing::debug!`, `print` for - debugging, with no other change. -- **Removing dead code** the user has already identified as unused. -- **Greenfield writes** — new file, new function inserted alongside - existing ones, new test case in a fresh test block. - -## Gray zone — use judgment - -When the change is genuinely ambiguous: - -- **"Add a test for this function"** — querying the function's prompt may - reveal what behavior the original turn intended to enforce, which informs - the test cases. Worth a query if the function has visible business logic; - skip for pure utility functions. -- **"Fix this bug"** — the bug fix may be trivial (off-by-one, null check) - or it may require understanding the original constraint. Read the diff the - user is asking you to fix; if the broken behavior is in AI-written code, - query. -- **"Make this faster"** — performance changes often preserve behavior, but - the original prompt may have called out a correctness/perf trade-off - ("don't cache — staleness matters more than latency"). Worth a query. -- **Configuration changes** — the `paths:` glob excludes `*.json`, - `*.yaml`, `*.toml`, etc. by default, but if the user asks about a - config-adjacent code file (e.g., a TS file that builds config), the - default substantive rules apply. - -## Cost of querying when you didn't need to - -Low. `prov log --only-if-substantial` returns empty quickly for short or -note-less files. Erring toward "query" is cheap; the cost is one extra -sub-50ms tool call. - -## Cost of skipping when you should have queried - -High. The agent may reintroduce a bug the original prompt explicitly -prevented, or rewrite a deliberate constraint as if it were an arbitrary -choice. The user typically doesn't know the constraint exists either — -that's why it lives in the prompt rather than a comment. - -When in doubt, query. +# Triggers — recognizing a provenance question + +This skill activates on user questions about the *origin*, *intent*, or +*history* of code that already exists. The lever for activation is the +user's phrasing, not the file or the kind of work in progress. This page +maps common phrasings to the right query. + +## Phrasings that should trigger + +### "Why does this code do X?" / "Why is this set to Y?" + +The user wants the *reason* behind a specific behavior. The originating +prompt usually carries the reason (compliance constraint, perf trade-off, +a hard-won bug fix). Query with a **point lookup** on the line that encodes +the behavior: + +```bash +prov log src/payments.ts:247 +``` + +If the user named a function instead of a line, run `grep` or open the +file to find the relevant line, then look it up. + +### "What was the prompt for this?" / "Show me the prompt that wrote this." + +Direct ask. Same query — point lookup if they referenced a specific line +or block; whole-file lookup if they said "this file": + +```bash +prov log src/payments.ts:247 +prov log src/payments.ts +``` + +### "Who wrote this?" / "Did the agent write this?" / "Which model?" + +The user wants attribution: human vs. AI, and which model. Run a point +lookup; the response includes `model`, `conversation_id`, `turn_index`, +and `blame_commit`. If `status` is `no_provenance`, fall back to +`git blame` and tell the user this looks human-authored or pre-dates +`prov install`. + +### "What's the history of this file?" / "What prompts have shaped this?" + +Whole-file lookup, most useful for files with several captured edits: + +```bash +prov log src/payments.ts +``` + +If the user wants AI-on-AI rewrites surfaced too, add `--history`. + +### "Has this line been hand-edited since the AI wrote it?" / "Is this drifted?" + +Point lookup. The `status` field answers directly: `unchanged`, `drifted`, +or `no_provenance`. A `drifted` response includes `blame_author_after` so +you can tell the user who edited it. + +### "Find prompts about X" / "Where did we decide to use X?" + +Cross-file search: + +```bash +prov search "rate limiting" +``` + +Returns matching prompts with the commits and files they touched. Pair the +results with `git log` if the user wants to walk the decision history. + +### "What session was this from?" / "What conversation produced this?" + +Point lookup. The response carries `conversation_id` and `turn_index`. The +user can use those to find the original transcript in their agent harness. + +## Phrasings that should NOT trigger + +The skill is only for *origin/intent/history* questions. Skip it for: + +- **Edit requests** — "refactor this", "rename X to Y", "extract this into + a function", "fix this bug". Even if the file is AI-authored, don't run + `prov log` preemptively. If the user follows up with "why did the + original prompt set X to Y", *then* it's a provenance question. +- **Greenfield asks** — "create a new file", "write a function that does + X". There's nothing to query. +- **Pure code-reading asks** — "what does this function do" (the answer is + in the code itself), "trace the call graph" (use grep/code reading), "is + this used anywhere" (use grep). These are about the *current code*, not + its origin. +- **Project-level questions** that aren't about specific code — "what's + this project about" (read the README), "how is the codebase organized" + (read the structure). + +If the user's phrasing is ambiguous between "what does this code do" and +"why does this code do X", lean on whichever interpretation they emphasize. +"Why" and "what for" lean provenance; "what" and "how" lean code-reading. + +## Gray-zone phrasings + +These genuinely could go either way; use judgment: + +- **"Explain this code"** — usually a code-reading ask, but if the code has + unusual constants or non-obvious branches, the originating prompt may + carry the explanation. Worth a quick point lookup; if `no_provenance`, + fall back to explaining from the code. +- **"What's the intent here?"** — usually leans provenance ("intent" maps + to the original prompt), but if the file has no captured notes, treat as + a code-reading ask. +- **"Is this still correct?"** — leans code-review, but a provenance check + can reveal whether the current behavior matches the original spec + (drift). Worth a point lookup if the user is questioning correctness. + +## How user follow-ups shift the work + +After you answer a provenance question, the user often asks a follow-up +that *is* an edit ("OK, then change the window to 60 days"). That follow-up +isn't itself a provenance question, but the context you just established +should inform the edit: + +- If the prompt called the value out as load-bearing ("compliance requires + ..."), flag the constraint before making the change. +- If the line was already `drifted`, the existing value may be a human + override worth preserving — ask before overwriting. +- If the prompt was an arbitrary default ("just pick a sensible window"), + the change is uncontroversial. + +That's not the skill firing again — it's you carrying the answer you +already produced into the next turn. diff --git a/plugin/skills/prov/tests/skill_smoke.md b/plugin/skills/prov/tests/skill_smoke.md index 53f3650..cd51d52 100644 --- a/plugin/skills/prov/tests/skill_smoke.md +++ b/plugin/skills/prov/tests/skill_smoke.md @@ -1,104 +1,152 @@ # Skill smoke test plan (manual) -These four behavioral scenarios are the load-bearing verification for U12. -They run against a real Claude Code session — there is no automated harness -that tests trigger fidelity. Run them after every meaningful edit to -`SKILL.md`'s `description:` or body. +These behavioral scenarios are the load-bearing verification for the +question-triggered shape of this skill. There is no automated harness that +tests trigger fidelity — run them against a real Claude Code session after +any meaningful edit to `SKILL.md`'s `description:` or body. ## Setup -1. Install the prov binary so it's on `PATH`. +1. Install the `prov` binary so it's on `PATH`. 2. Install this plugin (`/plugin install --plugin-dir /plugin` or marketplace). -3. Restart Claude Code so hooks reload. -4. Use a fixture repo seeded with prov notes — either: - - A real repo where you've used Claude Code for a few sessions, or - - The fixture under `crates/prov-cli/tests/fixtures/` extended with - a manually-crafted note via `git notes --ref=refs/notes/prompts add`. +3. Restart Claude Code so the skill registry reloads. +4. Use a fixture repo seeded with provenance notes — either a real repo + where Claude Code has been used for a few sessions, or the fixture under + `crates/prov-cli/tests/fixtures/` extended with a manually-crafted note + via `git notes --ref=refs/notes/prov add`. -## Scenario 1 — substantive ask triggers the skill +## Scenario 1 — "why" question on a specific line triggers the skill **Prompt to Claude Code:** -> Refactor `src/payments.ts` to extract the dedupe logic into a separate -> module. +> Why is `src/payments.ts:247` set to 90 days? **Expected behavior:** -- Agent calls `prov log src/payments.ts` (or `:`) before proposing - edits. -- Agent surfaces the prior dedupe-window prompt in its plan, e.g., - *"the originating prompt called for a 90-day window for compliance — I'll - preserve that."* -- Final edit preserves the load-bearing constraint. +- Agent runs `prov log src/payments.ts:247` (with or without `--json`). +- Agent surfaces the originating prompt verbatim in the answer. +- Answer names the load-bearing constraint (e.g., "compliance requires + 90-day dedupe") and attributes it to the model + session. -**Pass criteria:** the agent runs `prov log` and cites the prompt before -writing code. +**Pass criteria:** the agent ran `prov log` and quoted the prompt in its +answer. -## Scenario 2 — trivial single-line change does NOT trigger +## Scenario 2 — file-level history question triggers the skill **Prompt to Claude Code:** -> Fix the typo on line 12 of `README.md`. +> What's the prompt history of `src/payments.ts`? **Expected behavior:** -- Agent does not call `prov log`. -- The `paths:` glob excludes `*.md`, so the skill should not even surface. +- Agent runs `prov log src/payments.ts`. +- Agent renders the captured edits with prompts, models, and timestamps, + most recent first. -**Pass criteria:** no `prov log` invocation in the session log. +**Pass criteria:** the agent ran the whole-file query and listed at least +one prompt. -## Scenario 3 — greenfield does NOT trigger +## Scenario 3 — search question triggers the skill **Prompt to Claude Code:** -> Create a new file `src/utils/format.ts` with a function that formats a -> Date as `YYYY-MM-DD`. +> Find the prompts where we talked about rate limiting. **Expected behavior:** -- Agent does not call `prov log` — the file doesn't exist yet, so there's - no provenance to query. -- If the agent does call it, the response should be empty (no notes for a - non-existent file) and the agent should proceed without it. +- Agent runs `prov search "rate limiting"`. +- Agent renders the hits with the prompts and the files/commits they + touched. -**Pass criteria:** either the agent skips the query, or queries it and -correctly handles the empty response without surfacing it as a finding. +**Pass criteria:** the agent ran `prov search` rather than grepping the +codebase. -## Scenario 4 — drifted line surfaces drift state +## Scenario 4 — drifted line surfaces the divergence + +**Setup detail:** seed the fixture so line 247 has been hand-edited since +the original AI capture, so `prov log src/payments.ts:247 --json` returns +`status: "drifted"`. **Prompt to Claude Code:** -> Explain `src/payments.ts:247`. +> Why is `src/payments.ts:247` set to its current value? -**Setup detail:** ensure the line at 247 has been hand-edited since the -original AI capture, so `prov log src/payments.ts:247 --json` returns -`status: "drifted"`. +**Expected behavior:** +- Agent runs `prov log src/payments.ts:247`. +- Answer surfaces both the *original* AI prompt AND the divergence (the + current value differs from the AI capture, and `blame_author_after` + shows who changed it). +- Agent flags that the current value may be a deliberate human override. + +**Pass criteria:** the answer names both the original intent and the +drift. + +## Scenario 5 — edit request does NOT trigger the skill + +**Prompt to Claude Code:** +> Refactor `src/payments.ts` to extract the dedupe logic into a separate +> module. + +**Expected behavior:** +- Agent does NOT preemptively run `prov log`. +- Agent proceeds with the refactor; it may read the file, plan, and edit + as normal. + +**Pass criteria:** no `prov log` invocation appears in the session log +unless the user asked a provenance follow-up. + +This is the central regression the rewrite exists to prevent — the prior +shape of the skill ran `prov log` before any non-trivial edit, which +created unwanted context bloat and noise. + +## Scenario 6 — no-provenance case is reported plainly + +**Prompt to Claude Code:** +> Who wrote `src/utils.ts:5`? + +**Setup detail:** ensure `src/utils.ts:5` has no captured note (a +human-authored line in a fresh repo works). + +**Expected behavior:** +- Agent runs `prov log src/utils.ts:5`. +- Response is `status: "no_provenance"` (or similar). +- Agent answers plainly: "No provenance recorded for that line — likely + human-authored or predates `prov install`," optionally falling back to + `git blame`. + +**Pass criteria:** the agent neither invents an explanation nor treats the +empty response as an error. + +## Scenario 7 — greenfield / non-provenance question does NOT trigger + +**Prompt to Claude Code:** +> Create a new file `src/utils/format.ts` with a function that formats a +> Date as `YYYY-MM-DD`. **Expected behavior:** -- Agent calls `prov log src/payments.ts:247`. -- Agent's explanation references both the original prompt AND the drift - state, e.g., *"originally written by Claude in turn 4 of session - sess_abc123 against the prompt 'add a 90-day dedupe window'; the line - has since been hand-edited (drifted)."* +- Agent does NOT run `prov log` or `prov search`. +- Agent writes the file as requested. -**Pass criteria:** explanation surfaces both the original intent and the -divergence. +**Pass criteria:** no `prov` invocation in the session log. ## Iteration loop If a scenario fails: -- **Trigger fails for substantive asks (false negatives)** — the - `description:` field is the lever. Add more trigger phrasing - ("before refactoring", "before editing AI-written code", - "to recover the original prompt"). Re-test. -- **Trigger fires for trivial asks (false positives)** — strengthen the - "When NOT to use it" section in the body and add explicit exclusions to - `description:`. Re-test. -- **Drift state isn't surfaced** — the `references/querying.md` example - for drifted lines is the prompt the agent learns from. Make the example - louder. +- **Trigger fails for a provenance question (false negative)** — the + `description:` field is the lever. Add the missing phrasing pattern to + both `description:` and the "Phrasings that should trigger" section of + `references/triggers.md`. Re-test. +- **Trigger fires on an edit/refactor/greenfield ask (false positive)** — + the rewrite's central failure mode. Strengthen the "Phrasings that + should NOT trigger" section and reinforce the negative in + `description:`. Re-test until the false positive goes away. +- **Drift state isn't surfaced** — the example in `references/querying.md` + for drifted lines is the prompt the agent learns from. Make it louder + and more specific. ## Content lints (automated) -These lints run in CI via `crates/prov-cli/tests/cli_plugin_layout.rs`: +These lints run in CI via `crates/prov-cli/tests/cli_skill_layout.rs`: - `SKILL.md` exists and parses as YAML+Markdown. -- Frontmatter has `name` and `description` (both non-empty). +- Frontmatter has `name` and `description` (both non-empty; `description` + ≥ 60 characters). - Body is at most 500 lines. - `references/querying.md` and `references/triggers.md` exist and are referenced by name from `SKILL.md`. +- This smoke test plan exists at `tests/skill_smoke.md`. From 33433163c28dcf3adfba5ec56fbc6e77dce19415 Mon Sep 17 00:00:00 2001 From: Matt Fogel Date: Thu, 14 May 2026 08:59:06 -0400 Subject: [PATCH 2/2] =?UTF-8?q?restructure:=20drop=20plugin=20manifest,=20?= =?UTF-8?q?move=20skill=20to=20skills/,=20rename=20plugin/=20=E2=86=92=20a?= =?UTF-8?q?gent-hooks/?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Splits the two responsibilities that the Claude Code plugin layout conflated: capture hooks and the read-side skill. Each now has one canonical install path with no overlap, so users can't accidentally double-register hooks the way the prior `/plugin install prov` plus `prov install --agent claude` combination could. - Capture hooks: bundled at `agent-hooks/hooks.json`, still embedded into the `prov` binary and merged into `.claude/settings.json` by `prov install --agent claude`. The directory is the canonical home; the old `plugin/.claude-plugin/plugin.json` marketplace manifest is deleted. - Skill: moved to `skills/prov/` so the Vercel skills CLI auto-discovers it via the bare shorthand `npx skills add mattfogel/prov` (the CLI walks `skills//SKILL.md` from the repo root by default). - `prov install --plugin` flag and `print_plugin_instructions()` are gone, along with the `install_plugin_flag_does_not_touch_repo` test that asserted their behavior. - `cli_plugin_layout.rs` renamed to `cli_agent_hooks_layout.rs`; the plugin-manifest test is dropped; the hooks-bundle and README tests are updated to the new paths. - README updates: drop the "two install paths" framing on the plugin README (now `agent-hooks/README.md`); add the optional `npx skills add` step to the top-level quick start; replace the `plugin/` directory pointer with `agent-hooks/` and `skills/`. Smoke-test plan setup updated to describe installing the skill via `npx skills add` instead of the now-defunct `/plugin install` path. --- README.md | 11 ++- agent-hooks/README.md | 44 ++++++++++ {plugin/hooks => agent-hooks}/hooks.json | 0 crates/prov-cli/src/commands/install.rs | 40 ++------- ...in_layout.rs => cli_agent_hooks_layout.rs} | 59 ++++--------- crates/prov-cli/tests/cli_read.rs | 14 ---- crates/prov-cli/tests/cli_skill_layout.rs | 6 +- plugin/.claude-plugin/plugin.json | 19 ----- plugin/README.md | 84 ------------------- {plugin/skills => skills}/prov/SKILL.md | 0 .../prov/references/querying.md | 0 .../prov/references/triggers.md | 0 .../prov/tests/skill_smoke.md | 10 ++- 13 files changed, 84 insertions(+), 203 deletions(-) create mode 100644 agent-hooks/README.md rename {plugin/hooks => agent-hooks}/hooks.json (100%) rename crates/prov-cli/tests/{cli_plugin_layout.rs => cli_agent_hooks_layout.rs} (59%) delete mode 100644 plugin/.claude-plugin/plugin.json delete mode 100644 plugin/README.md rename {plugin/skills => skills}/prov/SKILL.md (100%) rename {plugin/skills => skills}/prov/references/querying.md (100%) rename {plugin/skills => skills}/prov/references/triggers.md (100%) rename {plugin/skills => skills}/prov/tests/skill_smoke.md (93%) diff --git a/README.md b/README.md index 310c3e2..8e995d6 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,14 @@ prov log src/auth.ts:42 # the originating prompt for one line prov search "rate limiting" # find prompts that mention rate limiting ``` +Optional: install the agent skill so Claude Code (or any harness that supports Anthropic-style Skills) can answer provenance questions directly in the session — "why does this line do X", "what's the history of this file", "is this drifted": + +```bash +npx skills add mattfogel/prov +``` + +The skill is independent of the capture hooks above — install it per-repo or globally as you prefer; see [Vercel's `skills` CLI](https://github.com/vercel-labs/skills) for flags. + Codex project-local hooks require Codex to trust the repo's `.codex/` config layer; review the installed hooks via `/hooks` before they run. By default, notes stay on your machine. Opt in to team sharing per-remote: @@ -71,7 +79,8 @@ Pre-1.0. The on-disk note format is stable; the CLI surface and config keys may - [`crates/prov-core`](crates/prov-core) — library: git wrapping, notes I/O, SQLite cache, redactor, transcript parsers. - [`crates/prov-cli`](crates/prov-cli) — the `prov` binary and subcommands. -- [`plugin/`](plugin) — Claude Code plugin (skill + hooks manifest). +- [`agent-hooks/`](agent-hooks) — Claude Code capture-hook bundle, embedded into the `prov` binary at build time. +- [`skills/`](skills) — the optional Anthropic-style agent skill (install via `npx skills add mattfogel/prov`). - [`codex/`](codex) — Codex hooks adapter. ## Contributing diff --git a/agent-hooks/README.md b/agent-hooks/README.md new file mode 100644 index 0000000..e0872d0 --- /dev/null +++ b/agent-hooks/README.md @@ -0,0 +1,44 @@ +# agent-hooks/ — Claude Code capture hooks + +`agent-hooks/hooks.json` is the Claude Code hook bundle that prov uses to +capture each turn's prompt and tool calls during a session. The file is +embedded into the `prov` binary at compile time and written into a repo's +`.claude/settings.json` by `prov install --agent claude`. + +Four hooks, each with a 5-second timeout: + +- `SessionStart` → `prov hook session-start` — capture the active model. +- `UserPromptSubmit` → `prov hook user-prompt-submit` — stage the prompt. +- `PostToolUse` (matched on `Edit|Write|MultiEdit`) → `prov hook post-tool-use` + — stage the edit. +- `Stop` → `prov hook stop` — mark the turn complete. + +The hooks only write to `/prov-staging/`; nothing they emit reaches +the agent's prompt. The staged content is flushed into a git note on +`refs/notes/prov` by the `post-commit` git hook (also installed by +`prov install`). + +## Install + +```bash +prov install --agent claude +``` + +That merges these entries into `.claude/settings.json` in the current repo +(idempotent — re-run safely after upgrades). Restart Claude Code so the +hooks reload. + +## Read surface + +The optional skill at [`../skills/prov/`](../skills/prov) teaches Claude +Code (or any harness that supports Anthropic-style Skills) to answer user +questions about provenance using `prov log` and `prov search`. Install it +separately with [Vercel's `skills` CLI](https://github.com/vercel-labs/skills): + +```bash +npx skills add mattfogel/prov +``` + +The skill is independent of these hooks — capture works fine without the +skill, and the skill works fine in any repo where the `prov` binary is on +`PATH` (whether or not these hooks are installed locally). diff --git a/plugin/hooks/hooks.json b/agent-hooks/hooks.json similarity index 100% rename from plugin/hooks/hooks.json rename to agent-hooks/hooks.json diff --git a/crates/prov-cli/src/commands/install.rs b/crates/prov-cli/src/commands/install.rs index f1da172..1029ceb 100644 --- a/crates/prov-cli/src/commands/install.rs +++ b/crates/prov-cli/src/commands/install.rs @@ -9,9 +9,6 @@ //! - Optionally adds prov's agent-harness hook entries to repo-local adapter config. //! - Initializes `/prov.db` and runs an initial reindex. //! -//! `--plugin` prints the Claude Code marketplace install command -//! (`/plugin install prov`) and exits without modifying the project's -//! `.claude/`. The plugin assumes the `prov` binary is on `PATH`. //! `--enable-push ` opts into team mode by adding the notes-tracking //! fetch refspec for the named remote. The pre-push gate that R6 promises //! ships in U8; until then `--enable-push` documents itself as "fetch only". @@ -47,9 +44,9 @@ const PRE_PUSH_TEMPLATE: &str = include_str!("../../../../githooks/pre-push"); /// pre-rewrite commit would orphan when git replaced the SHA. const POST_REWRITE_TEMPLATE: &str = include_str!("../../../../githooks/post-rewrite"); -/// Embedded plugin/hooks/hooks.json so `--plugin`-less installs can mirror the -/// plugin's hook entries into project-scope `.claude/settings.json`. -const PLUGIN_HOOKS_JSON: &str = include_str!("../../../../plugin/hooks/hooks.json"); +/// Embedded `agent-hooks/hooks.json` — the Claude Code capture-hook entries +/// merged into project-scope `.claude/settings.json` by `--agent claude`. +const CLAUDE_HOOKS_JSON: &str = include_str!("../../../../agent-hooks/hooks.json"); /// Embedded Codex hook template. Source: `codex/hooks/hooks.json`. const CODEX_HOOKS_JSON: &str = include_str!("../../../../codex/hooks/hooks.json"); @@ -61,10 +58,6 @@ pub const HOOK_BLOCK_END: &str = "# <<< prov"; #[derive(Parser, Debug)] pub struct Args { - /// Print the (currently pre-v1) plugin install instructions and exit - /// without modifying the repo. - #[arg(long)] - pub plugin: bool, /// Enable team-mode sync at install time (configures the fetch refspec for /// the named remote). Defaults to local-only — sync is opt-in per-repo. #[arg(long, value_name = "REMOTE")] @@ -89,11 +82,6 @@ enum AgentAdapter { } pub fn run(args: Args) -> anyhow::Result<()> { - if args.plugin { - print_plugin_instructions(); - return Ok(()); - } - let cwd = std::env::current_dir().context("could not read current directory")?; let git = Git::discover(&cwd).map_err(|e| match e { prov_core::git::GitError::NotARepo => anyhow!("not in a git repo"), @@ -187,22 +175,6 @@ fn adapters_label(adapters: &[AgentAdapter]) -> String { .join(", ") } -fn print_plugin_instructions() { - println!("Claude Code plugin install:"); - println!(); - println!(" 1. Install the prov binary so it's on PATH:"); - println!(" cargo install prov"); - println!(" # or: brew install mattfogel/tap/prov"); - println!(" # or: curl -fsSL https://raw.githubusercontent.com/mattfogel/prov/main/install.sh | sh"); - println!(); - println!(" 2. Inside Claude Code, run:"); - println!(" /plugin install prov"); - println!(); - println!(" (Note: `prov install --plugin` does not modify this project's"); - println!(" .claude/ directory. Use `prov install` without --plugin for the"); - println!(" per-repo install path that wires hooks and config locally.)"); -} - // -------- git config -------- fn configure_git(git: &Git) -> anyhow::Result<()> { @@ -312,12 +284,12 @@ pub(crate) fn claude_settings_path(git: &Git) -> PathBuf { } fn install_claude_settings(git: &Git) -> anyhow::Result<()> { - let plugin_hooks: Value = serde_json::from_str(PLUGIN_HOOKS_JSON) - .context("embedded plugin/hooks/hooks.json failed to parse")?; + let plugin_hooks: Value = serde_json::from_str(CLAUDE_HOOKS_JSON) + .context("embedded agent-hooks/hooks.json failed to parse")?; let plugin_hooks_obj = plugin_hooks .get("hooks") .and_then(Value::as_object) - .ok_or_else(|| anyhow!("embedded plugin hooks JSON missing top-level `hooks` object"))? + .ok_or_else(|| anyhow!("embedded agent-hooks/hooks.json missing top-level `hooks` object"))? .clone(); let path = claude_settings_path(git); diff --git a/crates/prov-cli/tests/cli_plugin_layout.rs b/crates/prov-cli/tests/cli_agent_hooks_layout.rs similarity index 59% rename from crates/prov-cli/tests/cli_plugin_layout.rs rename to crates/prov-cli/tests/cli_agent_hooks_layout.rs index 1ae69b4..b3d4d6a 100644 --- a/crates/prov-cli/tests/cli_plugin_layout.rs +++ b/crates/prov-cli/tests/cli_agent_hooks_layout.rs @@ -1,8 +1,10 @@ -//! U11 plugin-layout tests. +//! Agent-hooks bundle layout tests. //! -//! Validates the on-disk shape of `plugin/` against the documented Claude -//! Code plugin schema, plus the behavioral guarantee that -//! `prov install --plugin` exits without mutating the project's `.claude/`. +//! Validates the on-disk shape of `agent-hooks/` — the directory whose +//! `hooks.json` is embedded by `prov install --agent claude` into a repo's +//! `.claude/settings.json`. The previous shape lived under `plugin/` and +//! carried a Claude Code plugin manifest; the manifest is gone, but the +//! hooks bundle itself is still load-bearing and worth lint-testing. use std::path::{Path, PathBuf}; @@ -17,8 +19,8 @@ fn workspace_root() -> PathBuf { .to_path_buf() } -fn plugin_dir() -> PathBuf { - workspace_root().join("plugin") +fn agent_hooks_dir() -> PathBuf { + workspace_root().join("agent-hooks") } fn read_json(path: &Path) -> Value { @@ -31,37 +33,8 @@ fn read_json(path: &Path) -> Value { } #[test] -fn plugin_manifest_has_required_fields() { - let manifest = plugin_dir().join(".claude-plugin").join("plugin.json"); - let value = read_json(&manifest); - let obj = value - .as_object() - .expect("plugin.json must be a JSON object"); - - // The Claude Code plugin schema requires `name` at minimum; we additionally - // require `description` and `version` so the marketplace listing has - // enough metadata to render usefully. - for required in ["name", "description", "version"] { - assert!( - obj.contains_key(required), - "plugin.json is missing required field `{required}`" - ); - assert!( - obj[required].is_string() && !obj[required].as_str().unwrap().is_empty(), - "plugin.json field `{required}` must be a non-empty string" - ); - } - - assert_eq!( - obj["name"].as_str().unwrap(), - "prov", - "plugin name must be `prov` (matches binary name and marketplace install command)" - ); -} - -#[test] -fn plugin_hooks_register_all_four_events() { - let hooks_path = plugin_dir().join("hooks").join("hooks.json"); +fn agent_hooks_register_all_four_events() { + let hooks_path = agent_hooks_dir().join("hooks.json"); let value = read_json(&hooks_path); let hooks = value .get("hooks") @@ -119,7 +92,7 @@ fn plugin_hooks_register_all_four_events() { Some(*command), "event `{event}` command mismatch" ); - // Plan specifies a 5-second timeout for every capture hook. + // 5-second timeout for every capture hook. assert_eq!( nested_hook.get("timeout").and_then(Value::as_i64), Some(5), @@ -129,14 +102,10 @@ fn plugin_hooks_register_all_four_events() { } #[test] -fn plugin_readme_exists() { - let readme = plugin_dir().join("README.md"); +fn agent_hooks_readme_exists() { + let readme = agent_hooks_dir().join("README.md"); assert!( readme.exists(), - "plugin/README.md must exist so marketplace listings have install instructions" + "agent-hooks/README.md must exist so the directory is self-explanatory" ); } - -// Behavioral coverage for `prov install --plugin` not mutating `.claude/` -// lives in `cli_read.rs::install_plugin_flag_does_not_touch_repo` — the U5 -// install tests own that fixture setup and we don't duplicate it here. diff --git a/crates/prov-cli/tests/cli_read.rs b/crates/prov-cli/tests/cli_read.rs index 918ddd4..2b5d24d 100644 --- a/crates/prov-cli/tests/cli_read.rs +++ b/crates/prov-cli/tests/cli_read.rs @@ -395,20 +395,6 @@ fn install_heals_legacy_top_level_command_shape() { assert_eq!(stop_arr[0]["hooks"][0]["command"], "prov hook stop"); } -#[test] -fn install_plugin_flag_does_not_touch_repo() { - let tmp = init_repo(); - prov_in(tmp.path()) - .args(["install", "--plugin"]) - .assert() - .success() - .stdout(predicate::str::contains("/plugin install prov")); - - assert!(!tmp.path().join(".git/hooks/post-commit").exists()); - assert!(!tmp.path().join(".claude/settings.json").exists()); - assert!(!tmp.path().join(".codex/hooks.json").exists()); -} - #[test] fn install_enable_push_configures_fetch_refspec() { let tmp = init_repo(); diff --git a/crates/prov-cli/tests/cli_skill_layout.rs b/crates/prov-cli/tests/cli_skill_layout.rs index c03724a..c6e1aae 100644 --- a/crates/prov-cli/tests/cli_skill_layout.rs +++ b/crates/prov-cli/tests/cli_skill_layout.rs @@ -1,6 +1,6 @@ -//! U12 SKILL content lints. +//! SKILL content lints. //! -//! Validates the on-disk shape of `plugin/skills/prov/`: frontmatter has the +//! Validates the on-disk shape of `skills/prov/`: frontmatter has the //! required keys, the body fits under the 500-line cap, the two reference //! docs exist and are linked from `SKILL.md`, and the manual smoke-test plan //! is committed alongside. @@ -16,7 +16,7 @@ fn workspace_root() -> PathBuf { } fn skill_dir() -> PathBuf { - workspace_root().join("plugin").join("skills").join("prov") + workspace_root().join("skills").join("prov") } /// Splits a SKILL.md-style file into (frontmatter, body). Frontmatter is the diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json deleted file mode 100644 index 9ec036d..0000000 --- a/plugin/.claude-plugin/plugin.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "name": "prov", - "description": "Prompt provenance for Claude Code: capture each turn's prompt and edits as a git note on the resulting commit, then surface the originating prompt for any line via `prov log :`. Bundles the capture hooks (UserPromptSubmit, PostToolUse, Stop, SessionStart) and a Skill that teaches the agent to answer user questions about code provenance — origin, intent, history, drift — using `prov log` and `prov search`.", - "version": "0.1.1", - "author": { - "name": "Matt Fogel", - "email": "mattfogel@gmail.com" - }, - "homepage": "https://github.com/mattfogel/prov", - "repository": "https://github.com/mattfogel/prov", - "license": "MIT", - "keywords": [ - "provenance", - "git-notes", - "claude-code", - "ai-attribution", - "code-review" - ] -} diff --git a/plugin/README.md b/plugin/README.md deleted file mode 100644 index c601091..0000000 --- a/plugin/README.md +++ /dev/null @@ -1,84 +0,0 @@ -# prov — Claude Code plugin - -This directory is the [Claude Code plugin](https://code.claude.com/docs/en/plugins) shape -for [prov](https://github.com/mattfogel/prov), the prompt-provenance tool. It bundles: - -- `hooks/hooks.json` — registers four capture hooks on the Claude Code session - (`UserPromptSubmit`, `PostToolUse` matched on `Edit|Write|MultiEdit`, `Stop`, - `SessionStart`). Each hook calls `prov hook ` with a 5-second timeout. -- `skills/prov/SKILL.md` — teaches the agent to query its own prior reasoning - (`prov log :`, `prov search `) when the user asks about - code provenance ("why does this do X", "what was the prompt for this line", - "what's the history of this file"). The skill is question-triggered, not - edit-triggered — it does not preemptively query before refactors or edits. - -## Prerequisites - -The plugin assumes the `prov` binary is on `PATH`. Install it first via one of: - -```bash -cargo install prov # crates.io -brew install mattfogel/tap/prov # Homebrew tap -curl -fsSL https://raw.githubusercontent.com/mattfogel/prov/main/install.sh | sh # cosign-verified -``` - -Each release is Sigstore-signed; the curl-pipe-sh script verifies the signature before -exec. - -## Install paths - -You have two ways to wire prov into a repo. Pick one: - -### Option A — install the plugin - -Use the Claude Code plugin marketplace install: - -```text -/plugin install prov -``` - -Plugin install drops the hooks into Claude Code itself; capture works in every -repo where the binary is on `PATH`. - -### Option B — per-repo wiring - -If you don't want a global plugin install, run this inside each repo you care about: - -```bash -prov install --agent claude -``` - -`prov install` is idempotent. It writes: - -- `.git/hooks/post-commit`, `.git/hooks/pre-push`, `.git/hooks/post-rewrite` (chained - inside `# >>> prov` / `# <<< prov` delimiters so it composes with your own hooks). -- `.claude/settings.json` (merges this plugin's hook entries into the project-scope - Claude Code settings — same hook list as the marketplace install). -- `.git/prov.db` (SQLite read cache). - -Run plain `prov install` when you only want shared git hooks/cache and no agent -adapter config. Re-run `prov install --agent claude` after pulling a prov upgrade; -it self-heals legacy entries and reports installed adapters without duplicating config. - -## Verify it's working - -After install, restart Claude Code (so hooks reload), run a session, and commit. -Then: - -```bash -prov log : -``` - -should print the originating prompt for any line that came out of an AI edit. -If it returns "no provenance", check `.git/prov-staging/log` for capture errors — -hooks are non-blocking by design and write diagnostics there rather than crashing -the session. - -## Plugin metadata - -See `.claude-plugin/plugin.json` for the manifest. Hooks are auto-discovered from -`hooks/hooks.json`; skills are auto-discovered from `skills//SKILL.md`. - -## License - -MIT — same as the rest of the prov repo. diff --git a/plugin/skills/prov/SKILL.md b/skills/prov/SKILL.md similarity index 100% rename from plugin/skills/prov/SKILL.md rename to skills/prov/SKILL.md diff --git a/plugin/skills/prov/references/querying.md b/skills/prov/references/querying.md similarity index 100% rename from plugin/skills/prov/references/querying.md rename to skills/prov/references/querying.md diff --git a/plugin/skills/prov/references/triggers.md b/skills/prov/references/triggers.md similarity index 100% rename from plugin/skills/prov/references/triggers.md rename to skills/prov/references/triggers.md diff --git a/plugin/skills/prov/tests/skill_smoke.md b/skills/prov/tests/skill_smoke.md similarity index 93% rename from plugin/skills/prov/tests/skill_smoke.md rename to skills/prov/tests/skill_smoke.md index cd51d52..09b5911 100644 --- a/plugin/skills/prov/tests/skill_smoke.md +++ b/skills/prov/tests/skill_smoke.md @@ -7,9 +7,13 @@ any meaningful edit to `SKILL.md`'s `description:` or body. ## Setup -1. Install the `prov` binary so it's on `PATH`. -2. Install this plugin (`/plugin install --plugin-dir /plugin` or - marketplace). +1. Install the `prov` binary so it's on `PATH` and run + `prov install --agent claude` in the fixture repo (gets the capture + hooks in place so the fixture has notes to query). +2. Install this skill into the fixture repo: + `npx skills add github.com/mattfogel/prov` (or + `npx skills add ` when testing local + edits). 3. Restart Claude Code so the skill registry reloads. 4. Use a fixture repo seeded with provenance notes — either a real repo where Claude Code has been used for a few sessions, or the fixture under