From 382ff7bcd988368c1722131d58b8ea236c02df26 Mon Sep 17 00:00:00 2001 From: Chris Pezza Date: Sat, 18 Jul 2026 20:23:37 -0400 Subject: [PATCH] feat(ux-optimization): live-site audit and core-objective prompts (ADR 0014) Resolves ADR 0013's deferred live-site item and the missing objective artifact, both surfaced by the full quill-router deliverable pass: - 07_objective_metrics (core_objective; gap analysis now anchors on it) - 10_live_site_audit (Tier 2, context_gated; measure-don't-eyeball rules, both-themes/both-viewports, DOM verification before reporting blanks) - gap_analysis/optimization_spec renumbered 08/09 and wired to consume core_objective and live_site_audit - SKILL.md + CLAUDE.md tables, word limits, pilot-notes addendum, v0.3.0 Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 6 +- checks/word_limits.json | 2 + .../0014-live-site-audit-and-objective.md | 81 +++++++++++++++++++ .../references/ux-optimization-pilot-notes.md | 23 ++++++ plugin/.claude-plugin/plugin.json | 2 +- .../07_objective_metrics.md | 57 +++++++++++++ ...{07_gap_analysis.md => 08_gap_analysis.md} | 5 +- ...zation_spec.md => 09_optimization_spec.md} | 2 +- .../07_ux_optimization/10_live_site_audit.md | 51 ++++++++++++ plugin/skills/ux-optimization/SKILL.md | 10 ++- 10 files changed, 231 insertions(+), 8 deletions(-) create mode 100644 docs/adrs/0014-live-site-audit-and-objective.md create mode 100644 plugin/prompts/07_ux_optimization/07_objective_metrics.md rename plugin/prompts/07_ux_optimization/{07_gap_analysis.md => 08_gap_analysis.md} (95%) rename plugin/prompts/07_ux_optimization/{08_optimization_spec.md => 09_optimization_spec.md} (92%) create mode 100644 plugin/prompts/07_ux_optimization/10_live_site_audit.md diff --git a/CLAUDE.md b/CLAUDE.md index 6634821..6c5eb0a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,8 +85,10 @@ Entry point: `/optimize`. Audits an **existing** product: extracts artifacts fro | `07_ux_optimization/04_persona_extraction` | always | `proto_persona` | `initial_concept`, `user_flow` | | `07_ux_optimization/05_value_prop_synthesis` | always | `value_prop_inventory` | `initial_concept` | | `07_ux_optimization/06_validation_checkpoint` | always | `validation_log` | all extracted artifacts | -| `07_ux_optimization/07_gap_analysis` | always | `hypothesis_backlog` | `user_flow`, `proto_persona`, `value_prop_inventory` | -| `07_ux_optimization/08_optimization_spec` | always | `optimization_spec` | `hypothesis_backlog` | +| `07_ux_optimization/07_objective_metrics` | always | `core_objective` | `user_flow`, `proto_persona` | +| `07_ux_optimization/08_gap_analysis` | always | `hypothesis_backlog` | `user_flow`, `proto_persona`, `value_prop_inventory`, `core_objective` | +| `07_ux_optimization/09_optimization_spec` | always | `optimization_spec` | `hypothesis_backlog` | +| `07_ux_optimization/10_live_site_audit` | context_gated (deployed URL + browser; Tier 2) | `live_site_audit` | `user_flow` | After validation, the registry matches forward-pass Phases 00-04 and the standard downstream (product-flow, tech-spec, `/compile`) applies unchanged. Visual-layer defects are handed off to a design-system audit, not itemized in the spec. diff --git a/checks/word_limits.json b/checks/word_limits.json index bd7049d..7827db9 100644 --- a/checks/word_limits.json +++ b/checks/word_limits.json @@ -1,12 +1,14 @@ { "_comment": "Per-prompt word limits from the Pattern Guide Tier 1 table (docs/updates/00_ENHANCEMENT_PATTERN.md), keyed by frontmatter `name`. Counting rule per ADR 0009 Enforcement: body words excluding the block. Prompts absent here are uncovered, not checked.", "analyze-optimization-gaps": 300, + "audit-live-site": 350, "capture-initial-idea": 250, "compose-optimization-spec": 300, "create-problem-statement": 200, "create-proto-persona": 250, "define-api-endpoints": 500, "define-test-questions": 200, + "derive-core-objective": 300, "evaluate-hypothesis": 300, "explore-problem-space": 300, "extract-product-concept": 300, diff --git a/docs/adrs/0014-live-site-audit-and-objective.md b/docs/adrs/0014-live-site-audit-and-objective.md new file mode 100644 index 0000000..2f8d971 --- /dev/null +++ b/docs/adrs/0014-live-site-audit-and-objective.md @@ -0,0 +1,81 @@ +# ADR 0014: Live-Site Audit and Core Objective in the Reverse Pass + +## Status + +Accepted (2026-07-18) — owner-directed ("bake that functionality into the plugin") + +## Context + +ADR 0013 shipped the reverse pass with two known gaps: + +1. **Live-site auditing was explicitly deferred** ("as a Tier 2 branch; pilot + was code-only"). The full deliverable pass on the pilot target + (2026-07-18) then ran a browser audit of the deployed product and proved + the deferral was leaving value uncaptured: it upgraded every code-suspected + defect to a measured, confirmed finding (e.g., an 86px text overflow, a + 52,583px catalog page, a three-row mobile nav with no collapse) and caught + an entire class of render-only defects code review cannot see — headline + orphans, theme seams on pages mixing token systems, missing responsive + collapse, and the difference between a tooling artifact and a real blank + render (verifiable only in the DOM). +2. **No `core_objective` producer existed in the reverse sequence.** The + forward pass has one (Phase 02); the reverse pass jumped from validation + to gap analysis, leaving prioritization anchored on an implicit objective. + Packaging the pilot deliverables surfaced the hole: the strategy layer had + problem/personas/journeys/value-props/hypotheses but no formal objective + + metrics artifact, and the owner asked for the complete set. + +## Decision + +1. **Add `07_objective_metrics.md`** (produces `core_objective`, requires + `user_flow` + `proto_persona`, Tier 1, always). Runs after the validation + checkpoint; `08_gap_analysis` now requires and consumes `core_objective` + so priorities anchor on a named objective and its funnel metrics. The + prompt's key rule: when the product's positioning bet is unvalidated, the + objective must be chosen so its telemetry *tests* the bet rather than + assuming it. +2. **Add `10_live_site_audit.md`** (produces `live_site_audit`, requires + `user_flow`, **Tier 2**, context_gated on "deployed URL known + browser + tooling available"). Method rules encode the pilot lessons: measure rather + than eyeball (scrollWidth vs clientWidth, page heights, element counts); + audit both themes via the product's own theme mechanism and both desktop + and mobile viewports; verify blank/broken renders in the DOM before + reporting them; stay outside the authenticated boundary without owner + access. `09_optimization_spec` consumes `live_site_audit` when present, + citing its measurements and marking code-only claims as unverified-live. +3. **Renumber** `07_gap_analysis` → `08_gap_analysis` and + `08_optimization_spec` → `09_optimization_spec` to keep filename order + matching execution order. Prompt `name:` fields are unchanged, so word + limits, dependency checks, and registry history are unaffected. + +Tier placement: the live-site audit stays Tier 2 (consistent with ADR 0013's +Tier Behavior text and ADR 0006's default-lean Tier 1) — a code-only run +remains valid; the browser pass is offered whenever its gate is satisfied. + +## Consequences + +**Positive:** +- Reverse-pass strategy layer now matches the forward pass artifact-for-artifact through Phase 02 (problem, persona, objective) — the convergence rule (ADR 0013 §6) covers objectives too. +- Visual findings ship with measurements instead of impressions, and render-only defect classes are no longer invisible to the framework. +- The tooling-artifact rule (verify in DOM before reporting) prevents the audit's most likely false-positive class. + +**Negative:** +- The live-site prompt depends on browser tooling availability, which varies by environment — the gate must be honest, and headless environments silently lose Tier 2 coverage. +- Two more prompts to maintain; the sequence is now 10 files. + +## Enforcement + +- **Testable consequences:** + - TC-1: `plugin/prompts/07_ux_optimization/` contains 10 prompts numbered 01-10 with filename order matching the SKILL.md sequence table. + - TC-2: `08_gap_analysis.md` lists `core_objective` in `requires` and embeds `{{core_objective}}`. + - TC-3: `10_live_site_audit.md` is `tier: 2` and `context_gated` with a `run_when` naming both the deployed-URL and browser-tooling conditions. +- **Checks:** covered by the existing suite (frontmatter-v2, placeholder-resolvability, dependency-graph, filename-convention run over the new files automatically); no new check code required. +- **Not machine-checkable:** whether live-audit findings actually carry measurements; whether the authenticated-boundary rule is honored at run time; browser-tooling availability at the user's site. +- **Graduation log:** _(empty)_ + +## References + +- ADR 0013 (reverse pass — deferred item resolved here), ADR 0006 (tiers), + ADR 0001 (frontmatter), ADR 0012 (enforcement). +- Pilot evidence: `docs/references/ux-optimization-pilot-notes.md` (live-site + addendum forthcoming with this change). diff --git a/docs/references/ux-optimization-pilot-notes.md b/docs/references/ux-optimization-pilot-notes.md index cd7017a..5e0a496 100644 --- a/docs/references/ux-optimization-pilot-notes.md +++ b/docs/references/ux-optimization-pilot-notes.md @@ -104,6 +104,29 @@ honest gaps + lowered confidence; forced past a gate → no guesswork). No prompt changes needed from this round. All pre-ship tests are complete; remaining steps are ADR 0013 acceptance and merge. +## Live-site audit + full deliverable pass (2026-07-18 addendum) + +The pilot's final phase drove the deployed product in a real browser (desktop +1600px + mobile 375px, both themes) and produced the complete owner package +(strategy, spec, live audit, content drafts, refactor plan, manifest+zip). +Lessons now encoded in ADR 0014 and prompts 07/10: + +- **Rendering proves what code predicts** — every code-suspected defect was + confirmed with a measurement (86px overflow; 52,583px page; 3-row mobile + nav), and an entire render-only class surfaced (chip orphans, theme seam on + /synth, 28-pill walls, unlabeled weight numbers) that code review missed. +- **Tooling-artifact discipline is essential:** blank screenshots after + programmatic scroll looked exactly like a broken page; DOM inspection + proved the content present. The prompt now requires DOM verification + before reporting blank/broken renders. +- **The strategy layer needed an objective artifact** — packaging exposed + that the reverse pass had no `core_objective` producer; prompt 07 fills + the hole and gap analysis now anchors on it. +- **Deliverable packaging shape that worked:** README (human TLDR + reading + order) + manifest.yaml (file roles, artifact names, execution map from + spec items → fix locations) + numbered docs + strategy dir with + provenance frontmatter. Candidate for a future `/package` output mode. + ## Open questions for the ADR 1. Naming: `optimize-ux` (Chris's working name) vs `product-audit`; command `/audit` vs `/optimize`. diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 9b1708d..e6af3b3 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "product-dev", "description": "AI-assisted product development framework. Guides from vague idea to technical spec through structured UX research, hypothesis formation, and prototype planning.", - "version": "0.2.0", + "version": "0.3.0", "author": { "name": "clownware" }, diff --git a/plugin/prompts/07_ux_optimization/07_objective_metrics.md b/plugin/prompts/07_ux_optimization/07_objective_metrics.md new file mode 100644 index 0000000..69c349b --- /dev/null +++ b/plugin/prompts/07_ux_optimization/07_objective_metrics.md @@ -0,0 +1,57 @@ +--- +name: derive-core-objective +description: > + Derive the product's core objective and funnel metrics from validated + journeys and personas. Reverse-pass counterpart of identify-core-objective. +run: always +produces: core_objective +requires: [user_flow, proto_persona] +tier: 1 +--- + + +You are a product strategist naming the single outcome an existing product +lives or dies on, and the metrics that would prove progress toward it. The +objective must be safe under evidence uncertainty: when the product's core +positioning bet is unvalidated, choose an objective whose telemetry tests +the bet rather than assuming it. + + +Traced journeys: +{{user_flow}} + +Extracted personas: +{{proto_persona}} + +Produce the core objective: + +**Objective**: One sentence naming the outcome, anchored on the make-or-break journey. Every ranked persona should converge on it. + +**Primary metric**: A single measurable rate with a time window (e.g., activation within 24h). Not a vanity metric — it must move only when a real user succeeds. + +**Supporting metrics**: A funnel table — step, metric, currently-known friction at that step (cite the journey findings). + +**Guardrail metrics**: 1-2 signals that catch the objective being gamed or the experience degrading elsewhere. + +**Why this objective**: 2-3 sentences connecting it to the evidence grades — especially how it behaves if the product's unvalidated bets turn out wrong. + +Close with provenance frontmatter (`mode: gap-analysis`, `requires:`, `validation_status` inherited). + + +- Do NOT pick a metric the team can move without users succeeding (visits, signups alone) +- Do NOT anchor the objective on an unvalidated positioning bet — make the objective's telemetry test the bet +- Do NOT list more than one primary metric; funnel steps carry the rest +- Do NOT exceed 300 words + + + +**Objective**: A new visitor catalogs their real collection in the first session and comes back to it within a week. + +**Primary metric**: Activation — % of new accounts reaching 5+ real (non-sample) teas in session one. + +**Supporting metrics**: Land → intent: hero CTA click-through (friction: three competing headlines). Intent → first tea: entry completion (friction: 12-field form, scanner flagged off). First tea → 5: bulk-add usage. + +**Guardrail**: 7-day return rate; entries-per-minute (catches padding the count with junk data). + +**Why**: The freshness wedge is founder-recalled, never user-observed. Activation is safe either way — if freshness matters, activated users will set freshness dates; the telemetry settles the bet. + diff --git a/plugin/prompts/07_ux_optimization/07_gap_analysis.md b/plugin/prompts/07_ux_optimization/08_gap_analysis.md similarity index 95% rename from plugin/prompts/07_ux_optimization/07_gap_analysis.md rename to plugin/prompts/07_ux_optimization/08_gap_analysis.md index 3afcfbe..b90c1f0 100644 --- a/plugin/prompts/07_ux_optimization/07_gap_analysis.md +++ b/plugin/prompts/07_ux_optimization/08_gap_analysis.md @@ -5,7 +5,7 @@ description: > clustering symptoms under root causes and anchoring on the critical journey. run: always produces: hypothesis_backlog -requires: [user_flow, proto_persona, value_prop_inventory] +requires: [user_flow, proto_persona, value_prop_inventory, core_objective] tier: 1 --- @@ -26,6 +26,9 @@ Extracted personas: Value-prop analysis: {{value_prop_inventory}} +Core objective: +{{core_objective}} + Produce the hypothesis backlog: **Root-cause clusters**: Group every gap and finding under its structural cause (e.g., "no message hierarchy", "activation wall", "template drift"). Mark each cluster class-fix or instance-fix. diff --git a/plugin/prompts/07_ux_optimization/08_optimization_spec.md b/plugin/prompts/07_ux_optimization/09_optimization_spec.md similarity index 92% rename from plugin/prompts/07_ux_optimization/08_optimization_spec.md rename to plugin/prompts/07_ux_optimization/09_optimization_spec.md index c8cde4a..9d3cdc6 100644 --- a/plugin/prompts/07_ux_optimization/08_optimization_spec.md +++ b/plugin/prompts/07_ux_optimization/09_optimization_spec.md @@ -19,7 +19,7 @@ traces to a journey, a persona, and evidence — a reader should never wonder Hypothesis backlog: {{hypothesis_backlog}} -Also read from `.product-dev/artifacts/`: `existing_feedback.md` if present (pre-existing tactical feedback from the owner or a reviewer — slot every item into the priority structure rather than appending it as a list), and `validation_log.md` for outstanding unvalidated flags. +Also read from `.product-dev/artifacts/`: `existing_feedback.md` if present (pre-existing tactical feedback from the owner or a reviewer — slot every item into the priority structure rather than appending it as a list), `live_site_audit.md` if present (browser-verified findings — cite its measurements in item rationales and mark code-only claims as unverified-live), and `validation_log.md` for outstanding unvalidated flags. Produce the optimization spec: diff --git a/plugin/prompts/07_ux_optimization/10_live_site_audit.md b/plugin/prompts/07_ux_optimization/10_live_site_audit.md new file mode 100644 index 0000000..cd8e878 --- /dev/null +++ b/plugin/prompts/07_ux_optimization/10_live_site_audit.md @@ -0,0 +1,51 @@ +--- +name: audit-live-site +description: > + Verify code-suspected UX defects on the running product and sweep for + visual issues code cannot show — in a real browser, both themes, + desktop and mobile viewports. +run: context_gated +run_when: Deployed product URL is known and browser tooling is available +produces: live_site_audit +requires: [user_flow] +tier: 2 +--- + + +You are a UX auditor driving the deployed product in a real browser. Code +review predicts defects; only rendering proves them. Your discipline: +measure, never eyeball — and never mistake a tooling artifact for a product +defect, or vice versa. + + +Traced journeys: +{{user_flow}} + +Audit the pages on the primary journeys, each at a desktop and a mobile viewport, in every theme the product offers (use the product's own theme mechanism — toggle, localStorage key, or query param). + +**Per page:** +1. **Verify code-suspected defects live.** Each prediction from the code audit gets a verdict: confirmed (with measurement), not reproduced, or worse-than-predicted. +2. **Measure, don't describe.** Overflow via `scrollWidth` vs `clientWidth`; page length in pixels and screens; element counts (nav links, cards, pills); wrap/orphan behavior at the actual breakpoint. +3. **Sweep for render-only defects** code cannot show: headline orphans, clipped text, theme seams, missing responsive collapse, CTA/headline mismatches, unlabeled UI values. +4. **Check the console** for errors — distinguish CSS/layout defects from script failures. + +**Verification rules:** before reporting a blank or broken render, confirm in the DOM that content is actually absent — capture glitches are tooling artifacts, not findings. Stay outside authenticated areas unless the owner has provided access. + +**Output**: findings grouped by page, each tagged `[confirms ]` or `[NEW]`, with its measurement and viewport/theme. Close with provenance frontmatter (`mode: extracted`, `provenance:` pages × viewports × themes, `confidence:`, `validation_status: pending`). + + +- Do NOT report a visual claim without a measurement or element count behind it +- Do NOT audit past the unauthenticated boundary without owner-provided access +- Do NOT let code findings bias the sweep — render-only defects are this prompt's unique value +- Do NOT exceed 500 words + + + +**/cabinet (1280px + 375px, light + dark)** +- `[confirms code finding]` Freshness badge overflows its chip: content 214px in a 160px container at 375px (scrollWidth vs clientWidth). +- `[NEW]` At 375px the toolbar wraps to two rows with no collapse — 31% of first viewport before content. +- `[NEW]` Dark theme: brew-journal panel stays light — its stylesheet defines own tokens, no `data-theme` scope. +- Console clean on both pages. + +`mode: extracted · provenance: 2 pages × 2 viewports × 2 themes · confidence: high (all measured) · validation_status: pending` + diff --git a/plugin/skills/ux-optimization/SKILL.md b/plugin/skills/ux-optimization/SKILL.md index d0c44dc..15fbb80 100644 --- a/plugin/skills/ux-optimization/SKILL.md +++ b/plugin/skills/ux-optimization/SKILL.md @@ -31,8 +31,10 @@ Prompts live in `${CLAUDE_PLUGIN_ROOT}/prompts/07_ux_optimization/`: | 4 | `04_persona_extraction.md` | always | `proto_persona` | | 5 | `05_value_prop_synthesis.md` | always | `value_prop_inventory` | | 6 | `06_validation_checkpoint.md` | always | `validation_log` | -| 7 | `07_gap_analysis.md` | always | `hypothesis_backlog` | -| 8 | `08_optimization_spec.md` | always | `optimization_spec` | +| 7 | `07_objective_metrics.md` | always | `core_objective` | +| 8 | `08_gap_analysis.md` | always | `hypothesis_backlog` | +| 9 | `09_optimization_spec.md` | always | `optimization_spec` | +| 10 | `10_live_site_audit.md` | context_gated (Tier 2) | `live_site_audit` | **Step 2 gate**: "Target repo contains research, interview, marketing, or outreach material." When skipping: "No research or marketing material found — skipping evidence mining. The problem statement will be inferred from code and README only, at reduced confidence. If research exists elsewhere (notes, call transcripts), share it and we'll run this step." @@ -48,7 +50,9 @@ Prompts live in `${CLAUDE_PLUGIN_ROOT}/prompts/07_ux_optimization/`: **Validation (step 6) — in-chat, soft gate.** Present claims lowest-confidence-first; update each artifact's `validation_status`; write `validation_log`. If the user defers or isn't the owner, proceed — but unvalidated flags propagate downstream and must be stated in the final spec. -**Gap analysis and spec (steps 7-8) — in-chat.** The spec integrates `existing_feedback` if present and carries the visual-layer handoff line (design-system defects belong to a design-audit skill, not this spec). +**Objective, gap analysis, and spec (steps 7-9) — in-chat.** The objective anchors gap-analysis priorities; the spec integrates `existing_feedback` if present and carries the visual-layer handoff line (design-system defects belong to a design-audit skill, not this spec). + +**Live-site audit (step 10) — Tier 2, browser required.** When the deployed product is reachable and browser tooling exists, offer it after journey tracing (its findings feed the spec) or after the spec ships (as verification). Run it via a subagent or in-chat with browser tools; it upgrades code-suspected defects into measured, confirmed findings and catches render-only issues (orphans, theme seams, missing responsive collapse) that code review cannot see. ## Registry