Skip to content

Run errors not shown correctly in evaluation table #4660

@junaway

Description

@junaway

When an evaluation run fails because the OpenAI API key is empty, the evaluation table does not show the run error correctly.

To reproduce:

  1. Set an empty OpenAI API key.
  2. Create a prompt that uses an OpenAI model.
  3. Run an evaluation on that prompt.

Expected behavior:

  • The table shows the scenario failed to run because of the missing API key.
  • Cost and latency show -.

Actual behavior:

  • The output is empty.
  • The cost and latency columns show errors about dicts.

evaluation table showing empty output and invalid output errors

run details showing missing API key error and invalid outputs error

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions