fix(ai): expose billing, quote, catalog and contract tools to the in-product chat - #3176
Merged
Merged
Conversation
…product chat The eleven aiTools registry entries behind invoices, quotes, contracts and the product catalog had no tool() declaration in createBreezeMcpServer and no TOOL_TIERS entry, so BREEZE_MCP_TOOL_NAMES omitted them and the technician chat could never call them — asked to "list our invoices" the model replied it had no invoicing tool. External MCP clients could reach them the whole time via getToolDefinitions(), which is why nothing looked broken. Same failure mode as the BE-16 vulnerability tools (#2605). It also made #3130's TIER2_READONLY_TOOLS allowlist entirely dead: GuardrailCheck.readOnly is only consumed by the chat's preToolUse (aiAgentSdk.ts) — the external MCP route runs checkGuardrails but ignores the flag — so all nine names on that allowlist annotated a surface that could not call them. - Declares all eleven tools in createBreezeMcpServer, input shapes mirroring toolInputSchemas and descriptions mirroring the canonical aiTools* copies. - Adds them to TOOL_TIERS at tier 2, matching their aiTools registry tier. The nine reads now resolve readOnly:true and auto-execute with a Tier-2 audit row; manage_invoices / manage_contracts carry no read-only or TIER2_ACTIONS entry so every action still prompts, with the financially-final ones escalating to Tier 3 via TIER3_ACTIONS. - Adds contract tests pinning TIER2_READONLY_TOOLS to TOOL_TIERS, to a real tool() declaration, and to the registry tier, plus coverage that the two mutators still prompt. The three new reachability tests fail against the pre-fix tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploying breeze with
|
| Latest commit: |
2431e91
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://64b21384.breeze-9te.pages.dev |
| Branch Preview URL: | https://fix-ai-chat-billing-catalog.breeze-9te.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was broken
Eleven AI tools —
list_invoices,get_invoice,manage_invoices,list_quotes,get_quote,list_contracts,get_contract,manage_contracts,search_catalog,get_catalog_item,lookup_distributor_product— were registered in theaiToolsexecution registry (aiToolsBilling.ts,aiToolsQuotes.ts,aiToolsContracts.ts,aiToolsCatalog.ts) with schemas, permissions and tiers, but had notool()declaration increateBreezeMcpServerand noTOOL_TIERSentry.BREEZE_MCP_TOOL_NAMES = Object.keys(TOOL_TIERS)is theallowedToolslist handed to the SDK, andcreateSessionPreToolUserejects!TOOL_TIERS[toolName]as "Unknown tool" — so the in-product technician chat could never see or call any of them.User-visible symptom: ask the chat "list our invoices" and the model replies that it has no invoicing tool, then routes the question to a neighbouring tool. Verified manually in the running app.
External MCP clients could reach all eleven the whole time via
getToolDefinitions(), which is why the gap never surfaced as a broken integration.This is the identical failure mode to the BE-16 vulnerability tools (#2605) — the code comment documenting that incident sits ~40 lines above the new block.
Secondary finding —
TIER2_READONLY_TOOLSwas entirely deadGuardrailCheck.readOnly(#3130 / #3156) is consumed in exactly one place:createSessionPreToolUseinaiAgentSdk.ts:653. The external-MCP route (routes/mcpServer.ts:1190) calls the samecheckGuardrailsbut reads only.allowedand.tier— it never looks atreadOnly.So the nine read tools #3156 added to
TIER2_READONLY_TOOLSwere 100% dead code, not merely degraded: the only surface that honours the allowlist is the same surface that could not call the tools. Nothing failed; the allowlist simply claimed to be optimising prompts for tools the chat did not have.The fix
apps/api/src/services/aiAgentSdkTools.tstool()declarations for all eleven, placed after the org-lifecycle block. Input shapes mirrortoolInputSchemasexactly (that is the gateexecuteToolvalidates against, andz.object()strips unknown keys rather than rejecting). Descriptions are duplicated inline as for every other tool in the file; theaiTools*modules keep the canonical copy served to external MCP.TOOL_TIERSentries at tier 2 for all eleven, matching theiraiToolsregistry tier exactly. This matters becausecheckGuardrailsresolves the base tier from the registry, not fromTOOL_TIERS— a divergence would mean the two maps describe different tools.Traced end-to-end rather than assumed:
actiondiscriminator in their input, socheckGuardrailsskips every per-action table and returns{ tier: 2, requiresApproval: false, readOnly: true }(base tier 2 +TIER2_READONLY_TOOLSmembership). InpreToolUsethat hitstier === 2 && readOnlyAutoExec→ auto-execute with anai_tool_executionsrow stampedread_only_auto. Paused sessions and unmatched calls during an active plan still prompt, unchanged.manage_invoices/manage_contracts— included deliberately so the fix(ai-tools): validate MCP billing/contract/catalog payloads; fix void delete_draft return #3028/fix(ai-tools): validate MCP billing/contract/catalog payloads with shared Zod schemas; fix void delete_draft return #3098delete_draftfix is reachable from chat. Neither is on any read-only allowlist and neither has aTIER2_ACTIONSentry, so draft actions fall to base tier 2 withreadOnlyunset → the per-step prompt still fires.issue/void/record_payment/void_paymentandactivate/pause/resume/cancelalready sit inTIER3_ACTIONS→ tier 3,requiresApproval: true, durable action intent. Both paths are now asserted, not assumed.The contract test
apps/api/src/services/aiGuardrails.readonly.contract.test.tsgains aTIER2_READONLY_TOOLS is reachable by the chat (#3156)block asserting:TOOL_TIERSentry (the direct guard against this bug class);tool('<name>', ...)declaration inaiAgentSdkTools.ts, read from source — aTOOL_TIERSentry alone only allowlists themcp__breeze__name, which is the [AI] Vulnerability tools are never selected — model routes CVE questions to posture/patch tools instead #2605 half of the failure;TOOL_TIERStier agrees with theaiToolsregistry tier.Plus a
billing/contract mutators still prompt (#3156)block pinning the tier-2-no-readOnlyand tier-3-requires-approval resolutions above.Verified genuinely red pre-fix: with
aiAgentSdkTools.tsreverted, all three reachability tests fail (3 failed | 20 passed). The existingaiAgentSdkTools.mcpCoverage.test.tsalready covers the generalTOOL_TIERS→tool()direction; the new tests close theTIER2_READONLY_TOOLS→TOOL_TIERSedge it could not see.Verification
tsc --noEmit --project apps/api/tsconfig.json— clean (needsNODE_OPTIONS=--max-old-space-size=8192).eslinton both changed files — clean.vitest run src/services/aiGuardrails src/services/aiAgentSdk src/services/aiTools src/services/clientAiTools src/services/scriptBuilderTools— 106 files, 1588 tests passed.vitest run src/routes/mcpServer src/services/aiAgentSystemPrompt src/services/streamingSessionManager src/routes/clientAi— 40 files, 368 tests passed.Out of scope
manage_quotesandmanage_catalogare also registered in theaiToolsregistry and still absent from chat, so the chat can now read quotes and catalog items but not create or edit them. Left alone deliberately — outside this change's brief and unverified against the quote/catalog write paths. Worth a follow-up issue.🤖 Generated with Claude Code