Skip to content

Complete annotation QA, docs, and final polish - #311

Merged
johnnygreco merged 2 commits into
feature/trace-annotationsfrom
issue-306-annotation-qa-polish
Jun 17, 2026
Merged

Complete annotation QA, docs, and final polish#311
johnnygreco merged 2 commits into
feature/trace-annotationsfrom
issue-306-annotation-qa-polish

Conversation

@johnnygreco

Copy link
Copy Markdown
Owner

Summary

Complete the final trace annotation QA and polish pass for #306. This adds durable QA artifacts with nine committed screenshots, concrete MCP/API evidence files, final docs for human annotation, MCP agent annotation, and dataset export, plus focused event-level and accessibility regression coverage.

The only UI polish change is making the annotation drawer panel opaque so mobile modal content does not visually bleed through the transcript.

Closes #306

Validation

  • make generate-check -> passed
  • make fmt-check -> passed
  • make test -> passed
  • make build -> passed
  • make lint -> passed
  • npm run test:frontend -> passed
  • npm run test:e2e -> passed on rerun; the first run had one transient dashboard-search scroll timeout, with all annotation tests passing
  • npm run test:a11y -> passed
  • npm run test:visual -> passed after updating transcript visual baselines for the new annotation controls
  • npx playwright test tests/e2e/trace-annotations-qa.spec.ts --reporter=line -> passed and regenerated docs/qa/trace-annotations/images/
  • make install-local INSTALL_DIR="$HOME/.local/bin" -> passed
  • tmux restart from installed binary with /tmp/beacon-trace-annotations-dev.toml and curl -fsS http://localhost:4600/ >/dev/null -> passed
  • Browser smoke against installed server -> dashboard and a real transcript loaded with annotation controls
  • git diff --check -> passed

Risks

The committed QA screenshots are deterministic fixture captures, not live database screenshots. The installed-server smoke check covers the live binary and local dashboard separately. Backward compatibility was not required and was not added.

Reviewer Notes

Focus review on final product cohesion: docs accuracy, MCP/API evidence usefulness, QA artifact reproducibility, mobile drawer polish, and whether the added Playwright coverage closes the remaining event-level/accessibility gaps. QA report: docs/qa/trace-annotations/README.md. Screenshot directory: docs/qa/trace-annotations/images/.

@johnnygreco
johnnygreco force-pushed the issue-306-annotation-qa-polish branch from ebb8d3d to 0c5daf3 Compare June 17, 2026 11:16
@johnnygreco

Copy link
Copy Markdown
Owner Author

Review Record

Frozen target: feature/trace-annotations a6ae886883d835df50c848d975728ca5a8a50d09..issue-306-annotation-qa-polish 0c5daf3d2b0b31625eb04987659dd980cdbc8d8f
Diff command: git diff a6ae886883d835df50c848d975728ca5a8a50d09..0c5daf3d2b0b31625eb04987659dd980cdbc8d8f

Panel:

  • Descartes: product cohesion, user docs, QA report usefulness, no backward-compat framing -> clean after fixes
  • Linnaeus: UI quality, mobile layout, accessibility, visual snapshots, CSS polish -> clean after fixes
  • Maxwell: MCP/API annotation contract evidence, schemas, target IDs, export wording, privacy -> clean after fixes
  • Lovelace: test maintainability, determinism, generated artifact stability, CI/validation sufficiency -> clean after fixes
    Skipped Review-Standard lenses: none

Accepted findings fixed:

  • docs/mcp.md used target_type: "session" for a session-wide list_annotations example. Fixed to use target_type: null and clarified session-only behavior.
  • MCP QA evidence used an incorrect message open_ref shape and abbreviated request arguments. Fixed the returned open_ref shape and added required nullable fields.
  • MCP update/delete excerpts looked sequential while both showed revision 2. Added an explicit note that they are independent targeted contract-test excerpts.
  • Annotated-trace export evidence omitted the event annotation while discovery reported three annotations. Added the event annotation to export evidence.
  • QA evidence/screenshots used stale fixture wording and event IDs. Rebranded committed QA evidence to first-class trace annotation scenario IDs.
  • QA screenshot spec rewrote tracked PNGs during normal Playwright discovery and deleted the image directory before capture. Gated it behind BEACON_QA_CAPTURE=1 and removed the directory delete.
  • API screenshot limitation was underdocumented. QA report now states screenshots render fixture JSON and real endpoint coverage comes from listed Go handler tests.
  • API export evidence requested label=dataset:eval while the MCP message annotation only had dataset:train. Added dataset:eval to that annotation.
  • QA report published a local dev session ID. Removed the local identifier.

Rejected findings:

  • none

Validation after fixes:

  • go test ./internal/mcp -run 'TestTool(CreateAnnotationSupportsMessageTarget|ListAnnotationsMessageOpenRefRoundTripsFromCreate|UpdateAndDeleteAnnotationVerifyMessageScope)$' -> passed
  • go test ./internal/web -run 'TestAnnotationAPI(AnnotatedTracesListsSessionsAndTargets|ExportAnnotatedTracesIncludesContextAndDeleted|ExportAnnotatedTracesPaginatesAnnotations)$' -> passed
  • BEACON_QA_CAPTURE=1 npx playwright test tests/e2e/trace-annotations-qa.spec.ts --reporter=line -> passed
  • npx playwright test tests/e2e/trace-annotations-qa.spec.ts --reporter=line -> passed with 3 skipped and no tracked artifact rewrite
  • npx playwright test tests/e2e/dashboard.spec.ts --grep "creates transcript timeline event annotations" --reporter=line -> passed
  • npx playwright test tests/e2e/a11y.spec.ts --grep "annotation" --reporter=line -> passed
  • jq empty docs/qa/trace-annotations/mcp-annotation-workflow.json docs/qa/trace-annotations/annotated-trace-api-workflow.json -> passed
  • make generate-check -> passed
  • make fmt-check -> passed
  • npm run test:frontend -> passed
  • npm run test:e2e -> passed
  • npm run test:a11y -> passed
  • npm run test:visual -> passed
  • git diff --check -> passed
  • CI at head 0c5daf3d2b0b31625eb04987659dd980cdbc8d8f -> all checks passed: format, generated, govulncheck, npm-audit, frontend, dependency-review, test, lint, build, playwright-dashboard, playwright-accessibility

Final status: all rerun reviewers clean and CI green.

@johnnygreco
johnnygreco merged commit 75333db into feature/trace-annotations Jun 17, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant