fix(eval): make _EvalMetricResultWithInvocation.expected_invocation optional#5221
fix(eval): make _EvalMetricResultWithInvocation.expected_invocation optional#5221giulio-leone wants to merge 2 commits intogoogle:mainfrom
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Response from ADK Triaging Agent Hello @giulio-leone, thank you for your contribution! Before we can merge this pull request, you'll need to sign the Contributor License Agreement (CLA). You can do so by following the instructions in the "cla/google" check at the bottom of the pull request. Thanks! |
e757949 to
6668a79
Compare
|
Hi @giulio-leone, Thank you for your contribution! We appreciate you taking the time to submit this pull request. |
|
Hi @ankursharmas , can you please review this. |
…ptional conversation_scenario eval cases intentionally pass expected_invocation=None from local_eval_service (matching the public EvalMetricResultPerInvocation model), but the private _EvalMetricResultWithInvocation required a non-None Invocation, causing a pydantic ValidationError. Changes: - Make expected_invocation Optional[Invocation] with default None - Guard attribute access in _print_details when expected_invocation is None Fixes google#5214
b5831bc to
974ac70
Compare
|
Rebased onto the current Local revalidation on the rebased branch:
The production fix remains unchanged; this update only brings the PR back in line with the repo formatting checks. |
Summary
Fixes #5214 —
AgentEvaluator.evaluate()crashes with apydantic ValidationErrorwhen evaluatingconversation_scenarioeval cases because_EvalMetricResultWithInvocation.expected_invocationis typed as requiredInvocation, whilelocal_eval_serviceintentionally passesNonefor scenario-based cases.Changes
src/google/adk/evaluation/agent_evaluator.pyMake
expected_invocationoptional (line 93):This aligns with the public
EvalMetricResultPerInvocationmodel ineval_metrics.py:323.Guard downstream attribute access in
_print_details(lines 439–456):Added
if per_invocation_result.expected_invocation else Noneguards foruser_content,final_response, andintermediate_dataaccesses. Both_convert_content_to_textand_convert_tool_calls_to_textalready handleNonegracefully.Tests
Added
tests/unittests/evaluation/test_eval_metric_result_with_invocation.pywith 5 regression tests:None/ omitted / realexpected_invocation_get_eval_metric_results_with_invocationpropagatesNone_print_detailsdoes not crash withNoneexpected invocationAll 5 tests pass. Existing eval tests unaffected (2 pre-existing failures in
test_custom_metric_evaluator.pyconfirmed onmain).