Phase 0: contrast helper + report-only theme baseline (no visual change) - #2464
Phase 0: contrast helper + report-only theme baseline (no visual change)#2464mpaiva wants to merge 3 commits into
Conversation
Adds relativeLuminance/contrastRatio/rateContrast and colorValueToHex to theme/color-utils.ts (measurement only, no rendering change), plus unit tests and a report-only fixture that publishes a contrast baseline across the six built-in themes. No theme values change and no build gate is added. Refs bastani-inc#2314, bastani-inc#2463
Review fix (PR bastani-inc#2464): the Bg-suffix heuristic scored context-specific text tokens against the assumed canvas, reporting ratios users never see (e.g. Mocha userMessageText 11.53 on canvas vs 8.69 on userMessageBg) and omitting fallback-backed pairs like searchMatchText/searchMatchBg. The fixture now measures explicit rendered pairs (user/custom messages, tool boxes, search matches, selection) using the same optional-token fallbacks as theme-class.ts, and only sweeps true canvas foregrounds against the assumed background.
|
Scope note / next steps. This PR is deliberately Phase 0 only — the contrast helper, a report-only fixture, and published baselines. No theme values, components, or layouts change, and there's no build gate. Once this lands, Phase 1 follows separately, per the plan in #2314: before/after mockups against the new fullscreen |
|
Thanks, Marcelo. We’re starting a formal merge-readiness code review now, including current-main compatibility and the relevant checks. We’ll follow up here shortly with the review result and any required changes. |
flora131
left a comment
There was a problem hiding this comment.
Thanks, Marcelo — I ran a formal review of the current head against its base and current main. The WCAG helper math checks out, all 14 focused tests pass, the output is deterministic, and this PR does not change Atomic’s appearance or runtime rendering.
I’m requesting four changes before Phase 0 lands:
-
Test each color against the background where users actually see it. The current report treats several tokens as canvas text, but Markdown and syntax colors render inside
userMessageBgandcustomMessageBg; tool renderers place more colors inside pending, success, and error backgrounds; selected rows use additional foregrounds; and working-indicator colors are omitted. This matters in practice: some Catppuccin Latte combinations reported as passing against the canvas fail against their rendered message background. Please derive and assert the complete rendered surface-to-foreground mapping rather than falling back to the canvas for these tokens. -
Account for terminals that reduce colors to a 256-color palette. The report measures source hex values, while Atomic also supports production rendering through
fgAnsi()andbgAnsi()in 256-color terminals. That quantization changes some ratings from passing to failing. Please emit separately labelled truecolor and 256-color results, or clearly narrow the report’s scope. Any terminal foreground/background assumptions should be labelled with the exact assumed values. -
Remove UI pairs that no longer render on current
main. Transcript search was removed, sosearchMatchTextonsearchMatchBgis now an unused compatibility pair rather than a visible Atomic surface. Please refresh the rendered-pair list against currentmain; the tokens can remain loadable for compatibility without being counted in the visible baseline. -
Commit the baseline report as a Markdown artifact. The issue scope calls for the baseline numbers to be committed and posted. The PR currently prints the table only when the test runs. Please commit the generated table with its assumptions, color mode, generation command, and source revision, and add a synchronization assertion so the checked-in report cannot drift from the generator.
The package-test location and direct node:fs use are correct. Once these baseline issues are addressed, we can rerun the package checks and required CI for final approval. Thanks again for taking on this foundation work and for correcting the original foreground/background pairing.
Address the Phase 0 review (PR bastani-inc#2464): measure each color on the surface where it actually renders, cover 256-color quantization, drop the removed search pair, and commit the baseline so it cannot drift. - Derive the full rendered surface -> foreground mapping: assistant prose and chrome on the terminal canvas; Markdown/syntax inside userMessageBg and customMessageBg; toolTitle/toolOutput/diffs/muted inside tool pending/success/error backgrounds; text/muted/accent on selectedBg; and the working-indicator tones on the canvas. Context-specific tokens are never scored against the canvas. - Add color-utils.quantizeColorValueToHex(value, mode) and emit separately labelled truecolor and 256-color results, so fgAnsi()/bgAnsi() quantization that flips ratings is visible. Assumed terminal fg/bg are labelled with exact values and left unquantized (the terminal paints its own defaults). - Remove searchMatchText/searchMatchBg from the visible baseline; transcript search was removed from main and the tokens remain loadable only as a compatibility fallback. - Commit test/theme-contrast-baseline.md (assumptions, color mode, generation command, stable theme-source fingerprint) and assert it stays in sync with the generator (regenerate via UPDATE_CONTRAST_BASELINE=1). Refs bastani-inc#2314, bastani-inc#2463
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
| fgHex, | ||
| bgHex, | ||
| ratio, | ||
| rating: rateContrast(ratio), |
There was a problem hiding this comment.
Normal text is reported as AA-large
measureTheme() records whether a row is ordinary text, but rates every 3.0–4.49 ratio as AA-large without considering that kind. The generated report therefore presents normal-size rendered text—including toolOutput on toolSuccessBg (3.37) and mdLink on userMessageBg (3.52)—as an AA-large result even though those text surfaces require 4.5:1. Make the verdict kind-aware: ordinary text below 4.5 should be FAIL, while the 3.0 threshold remains applicable to non-text rows, or clearly label it as conditional on large text.
Artifacts
Narrow script that invokes the actual contrast baseline generator
- Authored TypeScript script imports the PR's `measureTheme()` and `rateContrast()` implementations, enumerates built-in themes and modes, and asserts/report prints ordinary-text AA-large rows. Takeaway: the proof exercises the exact report generator rather than a reimplementation.
Observed generator output showing ordinary text rated AA-large
- Captured successful command output for the narrow generator script, including all-row count and mdLink/toolOutput examples with expected normal-text failures. Takeaway: 300 ordinary-text rows below 4.5 are labeled AA-large by the running implementation.
Focused contrast test run passing
- Captured successful focused Vitest run for the contrast helpers and baseline generator tests: 2 files and 11 tests passed. Takeaway: the misleading report classification is accepted by the current focused test suite.
Ran code and verified through T-Rex
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/coding-agent/test/theme-contrast-baseline.helper.ts
Line: 218
Comment:
**Normal text is reported as AA-large**
`measureTheme()` records whether a row is ordinary text, but rates every 3.0–4.49 ratio as `AA-large` without considering that kind. The generated report therefore presents normal-size rendered text—including `toolOutput on toolSuccessBg` (3.37) and `mdLink on userMessageBg` (3.52)—as an AA-large result even though those text surfaces require 4.5:1. Make the verdict kind-aware: ordinary text below 4.5 should be `FAIL`, while the 3.0 threshold remains applicable to non-text rows, or clearly label it as conditional on large text.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
@flora131 Re: change #2 (256-color quantization). The baseline now emits separately-labelled truecolor and 256-color results, with the assumed terminal fg/bg stated exactly. Visual summary of the impact — the same rendered pair in both modes, showing where ratings degrade in a 256-color terminal: 606 rendered pairs/mode, 189 WCAG rating flips (172 degrading). The 256-color failures trace to a real |
Phase 0 of #2314, scoped per @flora131's request in #2463: establish a contrast baseline and tooling with zero change to Atomic's appearance or UX.
What this adds
theme/color-utils.ts—relativeLuminance,contrastRatio,rateContrast, andcolorValueToHex(reusing the WCAG math already inimpeccable'sscreenshot-contrast.mjs). Pure functions; no rendering path touched.What this deliberately does NOT do
Baseline (assumed backgrounds)
Foregrounds measured against an assumed canvas (
#1e1e1edark /#fffffflight);*Bgtokens against assumed text.""(terminal default) tokens are omitted as unknowable — making that gap visible is part of the point. Full table posted in #2463.Verification
npm run typecheck— cleannpm run test --workspace=@bastani/atomic -- theme-contrast-helpers theme-contrast-baseline theme-builtins— 13 passingbiome check— cleanRefs #2314
Figma reference
The Phase 0 documentation page (Atomic - Design System → Phase 0) — what shipped, the before/after (capability, not pixels), the baseline, and a clearly-labeled NOT shipped Phase 1 preview. Link is a Figma export (valid ~30 days):
Closes #2463
Follow-up finding: 256-color background collapse (tracked in #2550)
The 256-color measurement surfaces a genuine rendering defect (not a measurement error), which is exactly the point of reporting both modes. In 256-color terminals every near-neutral dark background collapses to
#5f5f5f, and the tool backgrounds snap to saturated cube corners (toolPendingBg → #00005f,toolSuccessBg → #005f00,toolErrorBg → #5f0000). That is why most 256-color rows readFAIL— e.g.mdCode on userMessageBg5.85 (AA) → 2.67 (FAIL).Root cause is the
spread < 10grayscale gate inrgbTo256()(color-utils.ts): slightly-tinted dark colors (spread 13–25) are forced onto the 6×6×6 cube even though the grayscale ramp is 30–60× closer (#343541 → #3a3a3arecovers that pair to 4.76).Out of scope for this measurement-only PR (no rendering change here). Tracked for a follow-up fix + baseline regeneration in #2550.
Mockup (Figma frame) — each row is the same rendered foreground/background pair in truecolor vs 256-color, showing where the WCAG rating degrades. Measurement only; the
rgbTo256()fix is intentionally not applied here and is tracked in #2550.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Greptile Summary
This change adds WCAG contrast utilities and a checked-in baseline for bundled terminal themes. The baseline currently reports ordinary text with contrast below the 4.5:1 requirement as
AA-large, which can make failing normal-text combinations appear acceptable.Confidence Score: 4/5
The contrast baseline can misstate accessibility compliance for normal rendered text, so it should be corrected before relying on the report for theme decisions.
One reproduced non-security defect remains in the report’s rating logic: normal text is evaluated using the large-text threshold.
Files Needing Attention: packages/coding-agent/test/theme-contrast-baseline.helper.ts; packages/coding-agent/src/modes/interactive/theme/color-utils.ts
What T-Rex did
Comments Outside Diff (1)
General comment
textwhose ratios are below the 4.5 normal-text threshold but whose visible rating isAA-large. The executed generator found 300 such rows across its six themes and two color modes, includingtoolOutput on toolSuccessBgat 3.37 andmdLink on userMessageBgat 3.52.rateContrast()has only a ratio argument and unconditionally maps every ratio >=3.0 and <4.5 toAA-large(packages/coding-agent/src/modes/interactive/theme/color-utils.ts:288-291).measureTheme()calculateskindindependently but does not use it to determine the rating (packages/coding-agent/test/theme-contrast-baseline.helper.ts:211-218).kind: "text"rows below 4.5 asFAIL; reserve the 3.0 threshold/rating for rows genuinely classified asnon-text(or clearly report the 3.0 result as a conditional large-text result rather than a pass). Add focused assertions that normal-textmdLinkandtoolOutputrows in [3.0, 4.5) are not reported as passing AA.Prompt To Fix All With AI
Reviews (3): Last reviewed commit: "test(theme): render-accurate contrast ba..." | Re-trigger Greptile