fix(verdict): render per-case stdout in Run modal (reverses BRAT #2) - #28
Merged
Merged
Conversation
Previously the Run verdict modal showed the same combined stdout under every test-case tab. The BRAT #2 fix (2026-06-05) treated `code_output` as a flat run-global blob because the leetnotion library's `formatTestOutput` joins the array on '\n' before exposing it (node_modules/@leetnotion/leetcode-api/lib/index.js:1887). That join made the field look line-per-element on the wire — but the wire shape is element-per-case. Verified live 2026-06-11 against leetcode.com with a 3-case Two Sum probe printing distinct CASE-MARKER strings. Response carried: total_testcases: 3 code_output: [3 elements, exact-length, no trailing pad] std_output_list: [4 elements, padded with trailing '', each ending '\n'] Both fields are per-case. `std_output_list` is the modern canonical name; `code_output` is a deprecated alias (per the leetcode-runner Java model's @deprecated annotation pointing to std_output_list). Changes: - Type std_output_list / expected_std_output_list on RunCheckResponse - Replace global asString(code_output) with splitOutput-driven per-case read inside renderActiveCase; prefer std_output_list, fall back to code_output for older response shapes - Strip trailing newline added by std_output_list elements - Replace BRAT #2 line-per-element regression tests with per-case wire-shape tests using the live probe payload as the fixture; pin that switching tabs swaps stdout and that empty padded slots render no Stdout section
When a Run hits a runtime error mid-execution (case N throws after case N-1 passed), LC's interpret_solution returns: status_code: 15 code_answer: ["[0,1]", ""] (truncated at the throwing case) compare_result: "100" (full arity, '0' for throw + skipped) total_testcases: 3 full_runtime_error: <stack trace> Pre-fix the renderer routed this to the per-case tabs path and showed Case 1 PASS / Case 2 FAIL / Case 3 FAIL with empty Output boxes — no stack trace anywhere. Verified live 2026-06-11 with a 3-case Two Sum probe that solved case 1 and raised on call #2. renderRunResult now detects mid-run RE (status_code 15 + error text + partial code_answer) and: - tags throwIdx = code_answer.length - 1 as 'threw' - tags i > throwIdx as 'skipped' (LC stops sequentially) - default-activates the throwing tab so the user lands on the trace - throwing tab replaces Output with an Error pre containing the full stack trace; keeps Input + Expected for context - skipped tab shows a "Not executed — an earlier case raised an exception." placeholder; keeps Expected (LC pre-computes it) - PASS tabs render normally (Input + Stdout + Output + Expected) Tab chips: PASS / FAIL / THREW (red, like fail) / SKIP (muted gray) CSS additions in styles.css mirror existing chip variants. 7 new vitest cases pin: default-active throwing tab, chip variants, stack-trace placement on the throwing tab, normal Output preserved on PASS tabs, "Not executed" placeholder on SKIP tabs, AI Debug button wiring, and a regression guard that the all-fail path still routes through hasRunErrorPayload's single-error-block layout.
User-facing text only — clearer and more conventional than 'THREW' for non-developer audiences. The internal state name (`'threw'`) and CSS class (`leetcode-verdict-case-chip--threw`) stay unchanged so the existing styles + tests keep working unchanged.
LikeSundayLikeRain
added a commit
that referenced
this pull request
Jun 12, 2026
Promote 1.3.1 from beta (1.3.1-beta.3) to stable. Code is unchanged from 1.3.1-beta.3 — only docs/chore commits differ. - manifest.json / package.json / package-lock.json -> 1.3.1 - versions.json: add 1.3.1 -> minAppVersion 1.12.7 (CI does not patch this) - CHANGELOG.md: backfill [1.3.0] (inline-widget milestone) and [1.3.1] (#24-#28) sections All 5 BRAT beta items resolved. CI green on the promoted commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Run verdict modal showed the same combined stdout under every test-case tab. Each tab should show only its own case's
print()output.Root cause
The BRAT #2 fix (2026-06-05) at
src/solve/verdictModalRenderer.tstreatedcode_outputas a flat run-global blob. The reasoning was: when LC returnscode_outputasstring[], each element looks like one line of combined stdout — verified againstnode_modules/@leetnotion/leetcode-api/lib/index.js:1887, whereformatTestOutputjoins the array on\n.The leetnotion library does join on
\n, but the wire shape is element-per-CASE, not per-line. The library normalization is what's flat; the underlying response isn't.Verification
Live probe against
leetcode.com/problems/two-sum/interpret_solution/on 2026-06-11 with a 3-case Python3 solution that printsf"CASE-MARKER nums={nums} target={target}"per case. Response:Both fields carry per-case stdout.
std_output_listis the modern canonical name;code_outputis a deprecated alias (thexuhuafeifei/leetcode-runnerJava model markscode_output@Deprecatedwith the comment "this field is not used anymore, please use std_output_list").Changes
src/solve/types.ts— typestd_output_list?: string[]andexpected_std_output_list?: string[]onRunCheckResponse(previously fell through the[k: string]: unknownindex signature).src/solve/verdictModalRenderer.ts—renderActiveCasereads per-case stdout viasplitOutput(res.std_output_list ?? res.code_output, arity)[activeIdx]with the trailing\nstripped. Thecode_outputfallback covers older or partial response shapes (e.g., the existingrun-sample.jsonfixture has emptycode_output: []despite carryingstd_output_list).tests/solve/verdictModalRenderer.test.ts— replaces the BRAT feat: v1.1 — Contest, AI Coach, and Preview #2 line-per-element regression tests with per-case wire-shape tests using the live probe payload. New assertions:\nfromstd_output_listis stripped for display.code_outputwhenstd_output_listis absent.Test plan
npx vitest run tests/solve/verdictModalRenderer.test.ts— 32/32 pass (4 new + 28 existing).npm run build(tsc --noEmit + esbuild prod) — green.npm run lint— green (1 unrelated unused-import warning).The full vitest run shows 5 timeouts in unrelated AI/preview test files (
tests/ai/clearKey.test.ts,tests/ai/probe-debounce.test.ts,tests/ai/reset-disclosures-command.test.ts,tests/preview/router.test.ts). These are pre-existing flakes under load — running them in isolation offmain(without my changes) reproduces the same flake-vs-pass behavior. Confirmed by stashing my diff, re-running just those files: all 38 tests pass in 3.45s.