fix: remove output schema restriction for tools with requires_confirmation#150
fix: remove output schema restriction for tools with requires_confirmation#150spichen wants to merge 2 commits into
Conversation
|
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA). To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
de5e8c5 to
454eedf
Compare
|
Thank you for signing the OCA. |
|
@spichen thank you for your contribution, this is a nice PR :) Could you please address the formatting issues and rebase? |
…ation The output schema validation in _ensure_checkpointer_and_valid_tool_config rejected tools that had requires_confirmation=True with typed outputs. This is unnecessary — the output schema is metadata for the LLM, not a runtime constraint. LangGraph does not enforce tool output schemas, and both the approved path (normal typed output) and rejected path (plain string) work fine as ToolMessage content. Resolves oracle#149 Signed-off-by: Salah Pichen <nh.salah@gmail.com>
336bd36 to
69cc5cf
Compare
Done. Could you run checks now please? |
sonleoracle
left a comment
There was a problem hiding this comment.
Thanks for the PR. I agree the current validation is too strict: requires_confirmation=True should not force consumers to erase useful typed output schemas.
I don’t think we should remove the check entirely, though. The tricky case is a Flow ToolNode with multiple outputs. On approval, a tool can return a dict like:
{"stdout": "hello", "stderr": "", "exit_code": 0}and that maps cleanly. But on rejection, _confirm_then() returns a single denial string. There is no principled way to map that string into multiple declared outputs like stdout, stderr, and exit_code.
In WayFlow, Agent Spec's reference runtime, its behavior is conservative here: it allows typed/multiple outputs, but for rejected confirmed tools in flows it raises by default because rejection may break structured flow outputs. It does not synthesize a fake multi output rejection dict.
So I think the fix should be:
- Allow typed output schemas.
- Keep approved multi-output tools working when they return a dict.
- Make rejection for multi-output Flow ToolNodes fail explicitly with a clear error.
That preserves the useful part of this PR without leaving rejection semantics undefined.
| ], | ||
| requires_confirmation=True, | ||
| ) | ||
| flow = _make_simple_flow_with_tool(ToolNode(name="bash_node", tool=server_tool)) |
There was a problem hiding this comment.
This test is currently failing because _make_simple_flow_with_tool() only wires the first output. I would suggest to either:
- use a single typed object output and assert exact output, or
- manually build a flow with all three output edges wired.
Resolves #149
Tools with
requires_confirmation=Trueand typed outputs get rejected by_ensure_checkpointer_and_valid_tool_config. This forces consumers to strip output schemas before loading, which loses information the LLM could use.The check exists because
_confirm_thenreturns a plain string on rejection, and the worry was it would conflict with a typed schema. It doesn't —ToolMessage.contentis always a string in LangChain. The schema is never validated at runtime. Both the approved path (normal output) and rejected path (denial string) work fine.The code already had a TODO acknowledging this was too strict:
# TODO: refine to only raise output property does not support stringThis PR removes the output schema validation entirely (keeps the checkpointer check) and adds a test with typed outputs to prove it works.