Skip to content

fix(ai): expose billing, quote, catalog and contract tools to the in-product chat - #3176

Merged
ToddHebebrand merged 1 commit into
mainfrom
fix/ai-chat-billing-catalog-tools
Aug 6, 2026
Merged

fix(ai): expose billing, quote, catalog and contract tools to the in-product chat#3176
ToddHebebrand merged 1 commit into
mainfrom
fix/ai-chat-billing-catalog-tools

Conversation

@ToddHebebrand

Copy link
Copy Markdown
Collaborator

What was broken

Eleven AI tools — list_invoices, get_invoice, manage_invoices, list_quotes, get_quote, list_contracts, get_contract, manage_contracts, search_catalog, get_catalog_item, lookup_distributor_product — were registered in the aiTools execution registry (aiToolsBilling.ts, aiToolsQuotes.ts, aiToolsContracts.ts, aiToolsCatalog.ts) with schemas, permissions and tiers, but had no tool() declaration in createBreezeMcpServer and no TOOL_TIERS entry.

BREEZE_MCP_TOOL_NAMES = Object.keys(TOOL_TIERS) is the allowedTools list handed to the SDK, and createSessionPreToolUse rejects !TOOL_TIERS[toolName] as "Unknown tool" — so the in-product technician chat could never see or call any of them.

User-visible symptom: ask the chat "list our invoices" and the model replies that it has no invoicing tool, then routes the question to a neighbouring tool. Verified manually in the running app.

External MCP clients could reach all eleven the whole time via getToolDefinitions(), which is why the gap never surfaced as a broken integration.

This is the identical failure mode to the BE-16 vulnerability tools (#2605) — the code comment documenting that incident sits ~40 lines above the new block.

Secondary finding — TIER2_READONLY_TOOLS was entirely dead

GuardrailCheck.readOnly (#3130 / #3156) is consumed in exactly one place: createSessionPreToolUse in aiAgentSdk.ts:653. The external-MCP route (routes/mcpServer.ts:1190) calls the same checkGuardrails but reads only .allowed and .tier — it never looks at readOnly.

So the nine read tools #3156 added to TIER2_READONLY_TOOLS were 100% dead code, not merely degraded: the only surface that honours the allowlist is the same surface that could not call the tools. Nothing failed; the allowlist simply claimed to be optimising prompts for tools the chat did not have.

The fix

apps/api/src/services/aiAgentSdkTools.ts

  • tool() declarations for all eleven, placed after the org-lifecycle block. Input shapes mirror toolInputSchemas exactly (that is the gate executeTool validates against, and z.object() strips unknown keys rather than rejecting). Descriptions are duplicated inline as for every other tool in the file; the aiTools* modules keep the canonical copy served to external MCP.
  • TOOL_TIERS entries at tier 2 for all eleven, matching their aiTools registry tier exactly. This matters because checkGuardrails resolves the base tier from the registry, not from TOOL_TIERS — a divergence would mean the two maps describe different tools.

Traced end-to-end rather than assumed:

  • Nine reads — no action discriminator in their input, so checkGuardrails skips every per-action table and returns { tier: 2, requiresApproval: false, readOnly: true } (base tier 2 + TIER2_READONLY_TOOLS membership). In preToolUse that hits tier === 2 && readOnlyAutoExec → auto-execute with an ai_tool_executions row stamped read_only_auto. Paused sessions and unmatched calls during an active plan still prompt, unchanged.
  • manage_invoices / manage_contracts — included deliberately so the fix(ai-tools): validate MCP billing/contract/catalog payloads; fix void delete_draft return #3028/fix(ai-tools): validate MCP billing/contract/catalog payloads with shared Zod schemas; fix void delete_draft return #3098 delete_draft fix is reachable from chat. Neither is on any read-only allowlist and neither has a TIER2_ACTIONS entry, so draft actions fall to base tier 2 with readOnly unset → the per-step prompt still fires. issue/void/record_payment/void_payment and activate/pause/resume/cancel already sit in TIER3_ACTIONS → tier 3, requiresApproval: true, durable action intent. Both paths are now asserted, not assumed.

The contract test

apps/api/src/services/aiGuardrails.readonly.contract.test.ts gains a TIER2_READONLY_TOOLS is reachable by the chat (#3156) block asserting:

  1. every member has a TOOL_TIERS entry (the direct guard against this bug class);
  2. every member has a real tool('<name>', ...) declaration in aiAgentSdkTools.ts, read from source — a TOOL_TIERS entry alone only allowlists the mcp__breeze__ name, which is the [AI] Vulnerability tools are never selected — model routes CVE questions to posture/patch tools instead #2605 half of the failure;
  3. the TOOL_TIERS tier agrees with the aiTools registry tier.

Plus a billing/contract mutators still prompt (#3156) block pinning the tier-2-no-readOnly and tier-3-requires-approval resolutions above.

Verified genuinely red pre-fix: with aiAgentSdkTools.ts reverted, all three reachability tests fail (3 failed | 20 passed). The existing aiAgentSdkTools.mcpCoverage.test.ts already covers the general TOOL_TIERStool() direction; the new tests close the TIER2_READONLY_TOOLSTOOL_TIERS edge it could not see.

Verification

  • tsc --noEmit --project apps/api/tsconfig.json — clean (needs NODE_OPTIONS=--max-old-space-size=8192).
  • eslint on both changed files — clean.
  • vitest run src/services/aiGuardrails src/services/aiAgentSdk src/services/aiTools src/services/clientAiTools src/services/scriptBuilderTools106 files, 1588 tests passed.
  • vitest run src/routes/mcpServer src/services/aiAgentSystemPrompt src/services/streamingSessionManager src/routes/clientAi40 files, 368 tests passed.

Out of scope

manage_quotes and manage_catalog are also registered in the aiTools registry and still absent from chat, so the chat can now read quotes and catalog items but not create or edit them. Left alone deliberately — outside this change's brief and unverified against the quote/catalog write paths. Worth a follow-up issue.

🤖 Generated with Claude Code

…product chat

The eleven aiTools registry entries behind invoices, quotes, contracts and the
product catalog had no tool() declaration in createBreezeMcpServer and no
TOOL_TIERS entry, so BREEZE_MCP_TOOL_NAMES omitted them and the technician chat
could never call them — asked to "list our invoices" the model replied it had no
invoicing tool. External MCP clients could reach them the whole time via
getToolDefinitions(), which is why nothing looked broken.

Same failure mode as the BE-16 vulnerability tools (#2605). It also made #3130's
TIER2_READONLY_TOOLS allowlist entirely dead: GuardrailCheck.readOnly is only
consumed by the chat's preToolUse (aiAgentSdk.ts) — the external MCP route runs
checkGuardrails but ignores the flag — so all nine names on that allowlist
annotated a surface that could not call them.

- Declares all eleven tools in createBreezeMcpServer, input shapes mirroring
  toolInputSchemas and descriptions mirroring the canonical aiTools* copies.
- Adds them to TOOL_TIERS at tier 2, matching their aiTools registry tier. The
  nine reads now resolve readOnly:true and auto-execute with a Tier-2 audit row;
  manage_invoices / manage_contracts carry no read-only or TIER2_ACTIONS entry so
  every action still prompts, with the financially-final ones escalating to
  Tier 3 via TIER3_ACTIONS.
- Adds contract tests pinning TIER2_READONLY_TOOLS to TOOL_TIERS, to a real
  tool() declaration, and to the registry tier, plus coverage that the two
  mutators still prompt. The three new reachability tests fail against the
  pre-fix tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying breeze with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2431e91
Status: ✅  Deploy successful!
Preview URL: https://64b21384.breeze-9te.pages.dev
Branch Preview URL: https://fix-ai-chat-billing-catalog.breeze-9te.pages.dev

View logs

@ToddHebebrand
ToddHebebrand merged commit 6a4be57 into main Aug 6, 2026
55 checks passed
@ToddHebebrand
ToddHebebrand deleted the fix/ai-chat-billing-catalog-tools branch August 6, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant