Skip to content

feat: tool logging (Option B), workflows track QA, and new advanced track#3

Open
3t-dev wants to merge 4 commits into
mainfrom
feature/tool-logging
Open

feat: tool logging (Option B), workflows track QA, and new advanced track#3
3t-dev wants to merge 4 commits into
mainfrom
feature/tool-logging

Conversation

@3t-dev

@3t-dev 3t-dev commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

What changed

Four commits delivering the tool-validation architecture (Option B from docs/discussion/architecture-alternatives.md), a QA pass of the workflows track, and a brand-new advanced track. Every exercise validated end-to-end.

1. Option B — hook-based tool logging (3e0c6c4)

  • hooks/hooks.json: plugin-bundled PostToolUse hook (matcher-scoped to Edit|Write|Read|Grep|Glob|Bash|Skill) — activates automatically when the plugin is enabled, no user settings edits
  • tools/log-tool.sh: POSIX-only stdin-JSON logger; records tool usage only for calls touching ~/.cclab/workspace/, appending to the exercise's .tool_log.jsonl
  • /cclab:reset now clears the tool log so stale tool usage can't pass process validation
  • cc-009 "Master the Edit Tool": first exercise that validates the process (Edit used, not Write) in addition to the artifact, with graceful guidance when logging is inactive

2. Workflows track QA hardening (3b9a012)

A reviewer-agent pass of wf-001..wf-008 against June 2026 Claude Code docs found and fixed:

  • Reset bug in 5 exercises: setup.sh didn't remove learner-created files, so reset silently left exercises solved
  • Cheat paths: wf-002 passed with .env rules in allow (the opposite of the lesson); wf-004 frontmatter checks satisfiable from body text; wf-007 accepted single-underscore MCP tool names
  • Content accuracy: Bash(npm *) word-boundary semantics, description as the auto-invocation trigger, Python 3 prereq for wf-007; verified argument-hint: and 0-indexed $0/$1 are correct per current docs (left unchanged)
  • Execute bits set on all setup/validate scripts

3. New advanced track — adv-001..adv-006 (076b708)

Six exercises (+ docs/curriculum/CURRICULUM-mvp3.md) that go one level deeper than workflows without duplicating it:

ID Title Deepens
adv-001 Specialist Squad subagent specialization: tools/disallowedTools/model/permissionMode
adv-002 Iron Gate blocking PreToolUse hooks — validator executes the learner's hook
adv-003 Server Smith authoring a stdio MCP server — validator speaks JSON-RPC to it
adv-004 Parallel Worlds two parallel worktrees, merged back to main
adv-005 Forked Context skills with named arguments + context: fork
adv-006 Plugin Architect capstone: package skill + bundled hook into a plugin

/cclab:start banner/completion and CLAUDE.md updated for 3 tracks / 23 exercises.

4. Docs (1488d0f)

Architecture discussion documents moved into docs/discussion/.

Why

Artifact-only validation couldn't verify how a learner worked ("use the Edit tool", "use Grep"). Plugin-bundled hooks solve this with zero install friction — and unblock the whole class of process-validated exercises the new tracks rely on. The workflows QA and advanced track complete the curriculum through MVP3.

How to test

# hook logger unit behavior (gates on workspace path, ignores everything else)
bash tools/log-tool.sh < <payload.json with cwd/file_path inside ~/.cclab/workspace/cc-009>

# any exercise end-to-end (same pattern for all 23)
bash exercises/fundamentals/cc-009/setup.sh
bash exercises/fundamentals/cc-009/validate.sh   # exit 1 (fresh)
# ...apply the reference solution from hints.md level 3...
bash exercises/fundamentals/cc-009/validate.sh   # exit 0

# full matrix already run: fresh-fail → solution-pass → cheat-fail → post-reset-fail
# for all 9 fundamentals + 8 workflows + 6 advanced exercises

🤖 Generated with Claude Code

3t-dev and others added 4 commits June 11, 2026 00:04
…cise cc-009

- hooks/hooks.json: plugin-bundled PostToolUse hook (matcher-scoped),
  activates automatically when the plugin is enabled
- tools/log-tool.sh: POSIX-only stdin-JSON logger; records tool usage
  only when a call touches ~/.cclab/workspace/, appends to the
  exercise's .tool_log.jsonl
- reset skill: clears .tool_log.jsonl so stale tool usage cannot pass
  process validation after a reset
- cc-009 'Master the Edit Tool': first exercise validating the process
  (Edit tool used) in addition to the artifact, with graceful guidance
  when tool logging is inactive

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
QA pass of wf-001..wf-008 against June 2026 Claude Code docs:
- setup.sh now removes learner-created artifacts (wf-001/003/005/006/008)
  so /cclab:reset actually restores the initial state
- wf-002: deny-scoped sensitive-file check (allowing .env no longer
  passes), rule-count enforcement, tool-pattern requirement
- wf-003: name value enforced, heading check, stricter step regex,
  description field explained as the auto-invocation trigger
- wf-004: frontmatter-scoped field checks (body text can't satisfy them);
  argument-hint and $0/$1 verified as current documented conventions
- wf-006: tools-line-scoped write-access check, non-empty results files
- wf-007: timestamp-server name accepted as documented, double-underscore
  MCP pattern enforced, type field required, Python 3 prereq noted
- execute bits set on all setup.sh/validate.sh

All 8 exercises re-tested end-to-end: fresh fail → solution pass →
cheat fail → post-reset fail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…m design

Six new exercises going deeper than the workflows track without
duplicating it:
- adv-001 Specialist Squad — subagent specialization (tools allowlist,
  disallowedTools, model override, permissionMode)
- adv-002 Iron Gate — blocking PreToolUse hook; validator EXECUTES the
  learner's hook against dangerous/safe payloads (exit 2 / exit 0)
- adv-003 Server Smith — author a stdio MCP server; validator speaks
  JSON-RPC to the learner's server and checks real responses
- adv-004 Parallel Worlds — two parallel worktrees merged back to main
- adv-005 Forked Context — skills with named arguments + context: fork
- adv-006 Plugin Architect — capstone: package skill + bundled hook into
  a distributable plugin (the same shape cclab itself uses)

Also: CURRICULUM-mvp3.md design doc, start skill banner/completion
updated for 3 tracks / 23 exercises, CLAUDE.md structure refreshed.

Every exercise tested end-to-end: fresh fail → reference solution pass →
cheat/broken-variant fail → post-reset fail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- architecture-alternatives.md: evaluation of plugin/hooks/CLI/MCP
  architectures; Option B (hooks-based tool logging) was chosen and
  implemented in this branch
- cca-exam-architecture-mapping.md: companion mapping document

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant