fix(frontend): surface invocation errors instead of dumping HTML 404 pages#4702
Open
mmabrouk wants to merge 1 commit into
Open
fix(frontend): surface invocation errors instead of dumping HTML 404 pages#4702mmabrouk wants to merge 1 commit into
mmabrouk wants to merge 1 commit into
Conversation
…pages When a playground or evaluator revision had no invocation URL, or the invoke endpoint returned an HTML 404 page instead of JSON, the run recorded the raw HTML as its output. An LLM-judge evaluator then scored a blob of HTML instead of reporting the error. The execution paths now return clear messages: "No invocation URL configured for this revision..." when the URL is empty, and "Service unreachable at <url> (HTTP <status>)" when the body is HTML. Consolidates the empty-URL message and the HTML detection into one shared helper in @agenta/entities/shared/execution/invocationErrors so all four fetch paths share the same logic.
dosubot
Bot
added
the
size:L
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
📝 WalkthroughSummary by CodeRabbitRelease Notes
WalkthroughA new shared ChangesInvocation error helpers and propagation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Railway Preview Environment
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
When an LLM-judge evaluator ran against a misconfigured revision, its output was a blob of HTML instead of a score or an error.
Two cases caused this. A revision with no invocation URL (empty
uriandurl) resolved the request to the web app origin, which returned the app's 404 page. A revision whose service was unreachable returned an HTML error page instead of JSON. In both cases the execution path stored the raw HTML page as the run output, so the judge scored the page.Changes
The execution paths now return a clear message instead of the HTML body.
No invocation URL configured for this revision (empty uri and url).Service unreachable at <url> (HTTP <status>).The empty-URL message and the HTML detection used to be copy-pasted across four fetch paths. They now live in one dependency-free helper,
@agenta/entities/shared/execution/invocationErrors, that every path calls:executionRunner.tsand the playground web worker short-circuit an empty URL and detect HTML error bodies.executionItems.tskeeps a resolved-empty URL empty instead of falling back to the web app origin, so the empty-URL guard can fire.runnable/utils.tsuses the HTML detector in both the runnable and the built-in evaluator fetch paths.The helper is a deep import (not the package barrel) so the web worker bundle does not pull in the rest of
@agenta/entities.Before, an HTML 404 surfaced as:
After:
JSON error bodies and plain-text errors still pass through unchanged.
Tests / notes
invocationErrors.test.tscovering HTML detection (doctype, leading tag, case and whitespace), the non-match cases (JSON, a tag later in the string, null/empty), and the message builders.@agenta/playgroundunit suite passes (89 tests).@agenta/entitiesand@agenta/playgroundtypecheck clean.pnpm lint-fixpasses.What to QA
detailstill shows that message.