fix: preserve zero-width graphemes in the cell round-trip - #151
fix: preserve zero-width graphemes in the cell round-trip#151alexispurslane wants to merge 1 commit into
Conversation
printString accumulates width-0 sequences (ZWNJ, ZWSP, combining marks decoded as separate width-0 sequences) into the pending cell, but the next visible cell overwrites them instead of prepending them, so the graphemes are silently dropped from rendered output. A decomposed accent like "e\u0301x" renders as "ex". Prepend the accumulated zero-width content to the next visible cell. Input order and cell widths are unchanged; only the dropped bytes are recovered. Fixes the loss observed when round-tripping content through a canvas (e.g. lipgloss layer compositing).
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #151 +/- ##
==========================================
+ Coverage 60.49% 60.52% +0.02%
==========================================
Files 52 52
Lines 6751 6751
==========================================
+ Hits 4084 4086 +2
+ Misses 2391 2389 -2
Partials 276 276 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
CI note: the conformance failures are pre-existing on
|
| Failing test | main nightly (run 31243567629, 06:17 today) | this PR (run 31256704853) |
|---|---|---|
FuzzRenderer/8e096e0ae397d854 |
fails | fails |
FuzzScreenShowsContent/seed#28-31,34-37 |
fails | fails |
FuzzRedrawResyncs |
fails (new crasher from tonight's fuzz job) | n/a (fuzz job is schedule-only) |
PR-only failures: none. Every failing seed in this PR's run fails identically on main's own nightly test job, which ran hours before this PR existed. The committed crashers in internal/conformance/testdata/fuzz/ fail on every pull_request until the renderer bugs behind them are fixed — related territory to #140/#148/#150 (wide-cell handling across resizes).
The root-module CI (unit tests, go test ./... on main + this branch) is green; the new TestStyledString cases are red without the fix and green with it.
|
My agent commented the solution before even asking me, sorry about that! But yes, this appears to be a pre-existing issue. |
|
if you rebase on main the conformance test should be clean now! |
Problem
printString(styled.go) drops zero-width graphemes when decomposing a styled string into cells. Width-0 sequences (ZWNJ U+200C, ZWSP U+200B, combining marks that decode as width-0) are accumulated into the pending cell in thedefaultbranch (cell.Content += string(seq)), but the next visible cell overwrites them:So any content containing a standalone width-0 grapheme loses it on the round-trip. A decomposed accent renders wrong:
"e\u0301x"comes back as"ex".Repro
Fix
Prepend the accumulated zero-width content to the next visible cell instead of overwriting it. Input order is preserved (the accumulated sequences precede the visible grapheme in the input), cell widths are unchanged, and the end-of-string flush already emits a trailing width-0 cell, so trailing graphemes survive too.
Test
Two cases added to
TestStyledString: ZWSP between two letters and a combining mark before a letter. Both fail onmainand pass with the fix.Context
Found while building a TUI editor: the canvas round-trip (lipgloss layer compositing) silently stripped ZWNJ and combining marks from editor content every frame. Note that #95 is adjacent (unknown escape sequences are dropped in the same
defaultbranch) but is a separate bug.