Skip to content

fix(workflow): preflight structured output schemas before agents start - #4

Merged
kky42 merged 4 commits into
mainfrom
fix/workflow-schema-preflight
Jul 31, 2026
Merged

fix(workflow): preflight structured output schemas before agents start#4
kky42 merged 4 commits into
mainfrom
fix/workflow-schema-preflight

Conversation

@kky42

@kky42 kky42 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Intent

The developer was fixing a bug where Codex-backend subagents with structured output schemas would fail only at the remote API level, after agents had already started and tokens were consumed. The root cause was that schemas passed via --output-schema did not comply with the Codex/OpenAI strict structured-output subset: every object layer required additionalProperties: false and all properties had to appear in required (with nullable types for optional fields instead of omission). The developer wanted a workflow preflight check — a static AST scan of all agent({ schema }) calls performed before any agent starts, so invalid schemas are rejected immediately with precise error paths (line number, column, JSON schema path), zero agents are launched, and zero tokens are consumed. Requirements included: validate nested objects recursively, report all violations in one pass, keep the implementation minimal and focused (no full JSON Schema engine or Ajv), do not silently auto-repair schemas (that would change caller semantics), retain a runtime defensive check as a safety net for dynamic schemas and headless callers, and update the prompts.ts schema examples to show a correct portable strict schema. The developer explicitly ruled out over-engineering and wanted the change scoped tightly to this preflight contract.

What Changed

  • Added assertPortableOutputSchema in src/workflow/output-schema.ts — a recursive validator that enforces the portable strict JSON Schema subset (every object requires additionalProperties: false and all properties in required; allOf/oneOf are rejected), reporting all violations with precise JSON path and source location in one pass.
  • Extended the AST-based static preflight in src/workflow/script-validation.ts to resolve top-level export const schema references (fixing a false-positive where valid exported schemas were silently skipped) and invoke schema validation before any agent launches, so invalid schemas abort the workflow with zero agents started and zero tokens consumed.
  • Updated src/prompts.ts schema examples and AGENTS.md contract to document the portable strict schema requirements (additionalProperties: false, nullable optional fields, no allOf/oneOf).

Risk Assessment

✅ Low: The round-2 change is two focused rejection guards for allOf/oneOf that exactly implement the user's instruction — no traversal logic, no new paths, no regressions.

Testing

Targeted tests for schema preflight (parseWorkflowScript, runWorkflow, and workflow tool integration) all pass. The allOf/oneOf rejection paths from the top commit lacked automated coverage; three assertions were added to the existing preflight test and verified passing. The zero-agents-launched guarantee, nested object validation, and precise JSON-path error messages are all confirmed working end-to-end.

Evidence: Schema preflight evidence — allOf/oneOf rejection + targeted test results

PASS allOf at root rejected error: $.allOf: allOf is not supported in the portable strict schema subset; use anyOf with nullable types instead PASS oneOf at root rejected error: $.oneOf: oneOf is not supported in the portable strict schema subset; use anyOf with nullable types instead PASS allOf nested inside properties rejected error: $.properties.item.allOf: allOf is not supported in the portable strict schema subset; use anyOf with nullable types instead PASS valid schema without allOf/oneOf accepted ✓ parseWorkflowScript > preflights static structured output schemas (incl. new allOf/oneOf assertions) ✓ runWorkflow > rejects invalid schemas before launching any agent (agentCount=0) ✓ workflow tool integration > rejects invalid schemas before starting any workflow subagent details.status=error, details.agentCount=0, message includes "no subagents were started" 76 passed (test/workflow.test.ts + test/workflow-integration.test.ts)

=== Schema preflight evidence — fix/workflow-schema-preflight ===

--- allOf/oneOf rejection (assertPortableOutputSchema, manually exercised) ---

  PASS  allOf at root rejected
         error: $.allOf: allOf is not supported in the portable strict schema subset; use anyOf with nullable types instead
  PASS  oneOf at root rejected
         error: $.oneOf: oneOf is not supported in the portable strict schema subset; use anyOf with nullable types instead
  PASS  allOf nested inside properties rejected
         error: $.properties.item.allOf: allOf is not supported in the portable strict schema subset; use anyOf with nullable types instead
  PASS  oneOf nested rejected
         error: $.properties.item.oneOf: oneOf is not supported in the portable strict schema subset; use anyOf with nullable types instead
  PASS  valid schema without allOf/oneOf accepted

5 passed, 0 failed

--- Targeted test suite (workflow.test.ts + workflow-integration.test.ts) ---

  ✓ parseWorkflowScript > preflights static structured output schemas
      covers: missing additionalProperties, missing required, nested object,
              invalid property schema, invalid type,
              allOf at root, oneOf at root, allOf nested (newly added assertions)
  ✓ parseWorkflowScript > requires schemas to be statically available during preflight
  ✓ runWorkflow > rejects invalid schemas before launching any agent  (agentCount=0)
  ✓ runWorkflow > validates dynamic schema options before launching the requested agent
  ✓ workflow tool integration > rejects invalid schemas before starting any workflow subagent
      details.status=error, details.agentCount=0,
      message includes "no subagents were started",
      message matches /schema preflight.*additionalProperties/i

76 passed (test/workflow.test.ts + test/workflow-integration.test.ts)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed ✅
  • ⚠️ src/workflow/output-schema.ts:94 - validateSchemaNode walks anyOf but silently skips allOf and oneOf, so an object schema nested inside either passes preflight without being checked for additionalProperties: false or complete required.
  • ⚠️ src/workflow/script-validation.ts:109 - collectTopLevelConstants only matches VariableDeclaration nodes; a top-level export const schema = {...} is an ExportNamedDeclaration and is silently skipped, causing a false-positive preflight error with a misleading message even when the schema itself is valid.

🔧 Fix: reject allOf and oneOf in portable strict schema preflight
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • npx vitest run test/workflow.test.ts test/workflow-integration.test.ts --reporter=verbose
  • npx vitest run test/workflow-integration.test.ts -t 'invalid schemas' — confirms agentCount=0 and 'no subagents were started' message for an invalid schema
  • npx vitest run test/workflow.test.ts -t 'preflights static' — covers missing additionalProperties, missing required, nested objects, allOf/oneOf at root and nested
  • npx vitest run test/workflow.test.ts -t 'rejects invalid schemas' — confirms zero-agent-launch guarantee at runWorkflow level
  • Direct Node.js invocation of assertPortableOutputSchema for allOf/oneOf root and nested rejection paths, verifying precise JSON-path error messages ($.allOf, $.properties.item.allOf, etc.)
  • npm run check — full suite (178 tests) passes on Node 22
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@kky42
kky42 merged commit 085788d into main Jul 31, 2026
2 checks passed
@kky42
kky42 deleted the fix/workflow-schema-preflight branch July 31, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant