Skip to content

(bug) Feedback sent via the API can't be filtered in observability #4654

@mmabrouk

Description

@mmabrouk

When I send user feedback on a trace through the API and then open the Observability filter, I expect to filter traces by that feedback (for example by score or comment). The feedback fields never show up in the filter, so there is no way to filter by them. This happens for every evaluator's feedback, not just API-created ones.

Steps to reproduce

  1. Send feedback on a trace:
curl -X POST "$HOST/api/simple/traces/" \
  -H "Authorization: ApiKey $KEY" -H "Content-Type: application/json" \
  -d '{"trace":{"data":{"outputs":{"score":1,"comment":"looks good"}},
        "references":{"evaluator":{"slug":"user-feedback"}},
        "links":{"invocation":{"trace_id":"<32hex>","span_id":"<16hex>"}}}}'
  1. Open Observability, open Filter, add an "Is Annotation" condition, pick the user-feedback evaluator, then open the Feedback field dropdown.
  2. Expected: score and comment are selectable feedback fields. Actual: the dropdown is empty.

Debugging info

The evaluator is created, and its stored output schema (from POST /api/preview/simple/evaluators/query) nests the fields under an extra outputs key:

{
  "type": "object",
  "properties": {
    "outputs": {
      "type": "object",
      "properties": { "score": {"type": "integer"}, "comment": {"type": "string"} }
    }
  }
}

To do

  • Fix the filter so it reads the evaluator's real feedback metrics (it currently reads a field that is always empty).
  • Fix how the frontend reads the feedback type so the nested schema is understood.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions