Skip to content

fix: remove output schema restriction for tools with requires_confirmation#150

Open
spichen wants to merge 2 commits into
oracle:mainfrom
spichen:fix/remove-confirmation-output-schema-validation
Open

fix: remove output schema restriction for tools with requires_confirmation#150
spichen wants to merge 2 commits into
oracle:mainfrom
spichen:fix/remove-confirmation-output-schema-validation

Conversation

@spichen
Copy link
Copy Markdown
Contributor

@spichen spichen commented Apr 8, 2026

Resolves #149

Tools with requires_confirmation=True and typed outputs get rejected by _ensure_checkpointer_and_valid_tool_config. This forces consumers to strip output schemas before loading, which loses information the LLM could use.

The check exists because _confirm_then returns a plain string on rejection, and the worry was it would conflict with a typed schema. It doesn't — ToolMessage.content is always a string in LangChain. The schema is never validated at runtime. Both the approved path (normal output) and rejected path (denial string) work fine.

The code already had a TODO acknowledging this was too strict:

# TODO: refine to only raise output property does not support string

This PR removes the output schema validation entirely (keeps the checkpointer check) and adds a test with typed outputs to prove it works.

@spichen spichen requested a review from a team April 8, 2026 16:49
@oracle-contributor-agreement
Copy link
Copy Markdown

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
The following contributors of this PR have not signed the OCA:

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.

@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Required At least one contributor does not have an approved Oracle Contributor Agreement. label Apr 8, 2026
@spichen spichen force-pushed the fix/remove-confirmation-output-schema-validation branch 2 times, most recently from de5e8c5 to 454eedf Compare April 8, 2026 16:59
@oracle-contributor-agreement
Copy link
Copy Markdown

Thank you for signing the OCA.

@oracle-contributor-agreement oracle-contributor-agreement Bot added OCA Verified All contributors have signed the Oracle Contributor Agreement. and removed OCA Required At least one contributor does not have an approved Oracle Contributor Agreement. labels Apr 8, 2026
@dhilloulinoracle
Copy link
Copy Markdown
Contributor

@sonleoracle

@sonleoracle
Copy link
Copy Markdown
Member

@spichen thank you for your contribution, this is a nice PR :)

Could you please address the formatting issues and rebase?

…ation

The output schema validation in _ensure_checkpointer_and_valid_tool_config
rejected tools that had requires_confirmation=True with typed outputs.
This is unnecessary — the output schema is metadata for the LLM, not a
runtime constraint. LangGraph does not enforce tool output schemas, and
both the approved path (normal typed output) and rejected path (plain
string) work fine as ToolMessage content.

Resolves oracle#149

Signed-off-by: Salah Pichen <nh.salah@gmail.com>
@spichen spichen force-pushed the fix/remove-confirmation-output-schema-validation branch from 336bd36 to 69cc5cf Compare April 27, 2026 08:01
@spichen
Copy link
Copy Markdown
Contributor Author

spichen commented Apr 27, 2026

@spichen thank you for your contribution, this is a nice PR :)

Could you please address the formatting issues and rebase?

Done. Could you run checks now please?

Copy link
Copy Markdown
Member

@sonleoracle sonleoracle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I agree the current validation is too strict: requires_confirmation=True should not force consumers to erase useful typed output schemas.

I don’t think we should remove the check entirely, though. The tricky case is a Flow ToolNode with multiple outputs. On approval, a tool can return a dict like:

{"stdout": "hello", "stderr": "", "exit_code": 0}

and that maps cleanly. But on rejection, _confirm_then() returns a single denial string. There is no principled way to map that string into multiple declared outputs like stdout, stderr, and exit_code.

In WayFlow, Agent Spec's reference runtime, its behavior is conservative here: it allows typed/multiple outputs, but for rejected confirmed tools in flows it raises by default because rejection may break structured flow outputs. It does not synthesize a fake multi output rejection dict.

So I think the fix should be:

  1. Allow typed output schemas.
  2. Keep approved multi-output tools working when they return a dict.
  3. Make rejection for multi-output Flow ToolNodes fail explicitly with a clear error.

That preserves the useful part of this PR without leaving rejection semantics undefined.

],
requires_confirmation=True,
)
flow = _make_simple_flow_with_tool(ToolNode(name="bash_node", tool=server_tool))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is currently failing because _make_simple_flow_with_tool() only wires the first output. I would suggest to either:

  • use a single typed object output and assert exact output, or
  • manually build a flow with all three output edges wired.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LangGraph adapter rejects typed outputs on tools with requires_confirmation

3 participants