Skip to content

fix(prompts): consolidate routing guidance and correct runtime claims - #5

Merged
kky42 merged 2 commits into
mainfrom
fm/pi-flow-prompt-behavior-opt
Aug 1, 2026
Merged

fix(prompts): consolidate routing guidance and correct runtime claims#5
kky42 merged 2 commits into
mainfrom
fm/pi-flow-prompt-behavior-opt

Conversation

@kky42

@kky42 kky42 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Intent

Optimize pi-flow's coordinator prompt design using the approved behavior baseline and real Pi-session evidence while preserving the public Agent and workflow contracts. Keep routing profile-agnostic: root-direct work, focused or small flat direct Agent delegation, and workflow for saved, staged, structured, replay, or larger orchestration; reuse session_key only for the same logical child stream. Treat focused delegation as a soft expectation and do not encode or test profile names, models, specialist selection, or defaults. Consolidate duplicated prompt guidance, make injected guidance reflect only active tools, correct dynamic-schema and saved-workflow runtime claims, replace prose-driven synthetic routing tests with deterministic behavior anchors, and add a privacy-safe opt-in real-model routing regression path and honest before/after evidence. Preserve unrelated runtime APIs and behavior, never include private session history, credentials, or machine-specific paths, and ship through review, tests, documentation validation, push, PR, and green CI without merging.

What Changed

  • Consolidated and de-duplicated coordinator prompt guidance in src/prompts.ts; injected sections now reflect only the tools actually active (agent-only, workflow-only, both, or neither), correcting incorrect dynamic-schema and saved-workflow filename claims that were in the prior prompt
  • Replaced synthetic prose-driven routing tests (test/proactive-routing.test.ts) with deterministic behavior anchors covering tool-gating, profile deduplication, and session-key continuation contract across test/agent-contract.test.ts and the new test/delegation-scenarios.test.ts
  • Added a privacy-safe opt-in real-model routing regression driver (scripts/e2e/prompt-routing.mjs) with committed before/after evidence (scripts/e2e/prompt-routing-evidence.md) documenting the accepted routing boundary and variance limitations

Risk Assessment

✅ Low: The change is a well-bounded prompt-text refactor and test realignment: it consolidates two builders into one, gates guidance on active tools (verified by deterministic unit tests), corrects the dynamic-schema and filename claims that were previously wrong, and replaces synthetic prompt-matching routing tests with behavioral anchors — with no changes to any runtime API or execution path.

Testing

Ran the two test files that directly exercise the changed prompt-building logic (11 tests across agent-contract and delegation-scenarios suites) — all passed. Supplemented with a direct behavioral snapshot of buildFlowPrompt confirming every key user-intent constraint: routing boundary present exactly once, correct tool-gating, corrected dynamic-schema and filename claims, and accurate session_key guidance.

Evidence: Targeted test run + behavioral anchor verification
=== TARGETED TEST RUN — 372c54a (refactor: simplify prompt routing guidance) ===

Files exercised: test/agent-contract.test.ts, test/delegation-scenarios.test.ts
Runner: vitest v3.2.4 — verbose

 ✓ delegation scenario execution > allows narrow work to finish in the root without delegation
 ✓ delegation scenario execution > executes a small flat fan-out as fresh parallel Agent calls
 ✓ delegation scenario execution > continues one logical child stream with the same session key
 ✓ pi-subagent agent contract > registers the Claude-style Agent tool contract without profile-specific guidance
 ✓ pi-subagent agent contract > marks description and prompt required while keeping routing fields optional
 ✓ pi-subagent agent contract > loads as a pi package extension from package metadata
 ✓ pi-subagent agent contract > injects one profile roster and the approved routing boundary
 ✓ pi-subagent agent contract > retains detailed guidance with a custom base system prompt
 ✓ pi-subagent agent contract > appends detailed guidance only for active pi-flow tools
 ✓ pi-subagent agent contract > advertises saved workflows only while workflow is active
 ✓ pi-subagent agent contract > registers Agent when loaded through additionalExtensionPaths

 Test Files  2 passed (2)
      Tests  11 passed (11)
   Duration  964ms

=== BEHAVIORAL ANCHOR VERIFICATION (direct buildFlowPrompt snapshot) ===

ROUTING POLICY CONSTANTS:
  DIRECT_WORK_POLICY: Narrow, local work can stay in the root when delegation adds no value.
  AGENT_USE_POLICY:   Use Agent for one focused delegated task or a small flat fan-out of independent work.
  WORKFLOW_USE_POLICY: Use workflow for a saved workflow, dependent stages or control flow, structured results or branching, replay, or larger fan-out.

BOTH ACTIVE — sections present:
  has ## Agent:                   true
  has ## Workflow:                true
  has routing boundary section:   true
  profile roster in prompt (once): true  [general-purpose]
  'Available agents:' count = 1:  true  (deduplication confirmed)

AGENT ONLY — workflow section absent:
  has ## Agent:                   true
  has ## Workflow:                false
  AGENT_USE_POLICY injected:      true
  WORKFLOW_USE_POLICY absent:     true

WORKFLOW ONLY — agent section absent:
  has ## Agent:                   false
  has ## Workflow:                true

CORRECTED RUNTIME CLAIMS:
  dynamic-schema correction present:  true  ("schema supplied through dynamic options is validated immediately before that child launches")
  old static-only claim removed:      true  ("Schemas must be static" absent)
  filename-mismatch correction:       true  ("filename need not match `meta.name`")
  old exact-match claim removed:      true  ("use a filename that exactly matches" absent)

SESSION_KEY GUIDANCE:
  parallel-branches guidance:     true  ("independent work, including parallel branches, stays fresh")
  logical-child-stream anchor:    true  ("same logical child stream")

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ℹ️ src/prompts.ts:87 - buildFlowPrompt is exported but has no internal guard for the { agentActive: false, workflowActive: false } case. If called with both false it emits a # Subagent Delegation header with a profile roster but no tool-specific guidance sections — a semantically incomplete fragment. The only production call site (pi-subagent.ts:708) is correctly guarded and the test for neither passes through the early-return in the handler rather than through buildFlowPrompt itself.
✅ **Test** - passed

✅ No issues found.

  • npx vitest run test/agent-contract.test.ts test/delegation-scenarios.test.ts --reporter=verbose
  • Direct buildFlowPrompt snapshot: verified routing policy constants, section gating (both/agent-only/workflow-only/neither), profile deduplication, corrected dynamic-schema claim, corrected saved-workflow filename claim, and session_key parallel-branches guidance
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@kky42
kky42 merged commit 325ad60 into main Aug 1, 2026
2 checks passed
@kky42 kky42 mentioned this pull request Aug 1, 2026
@kky42
kky42 deleted the fm/pi-flow-prompt-behavior-opt branch August 2, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant