Skip to content

fix: preserve zero-width graphemes in the cell round-trip - #151

Open
alexispurslane wants to merge 1 commit into
charmbracelet:mainfrom
alexispurslane:fix/zero-width-grapheme-loss
Open

fix: preserve zero-width graphemes in the cell round-trip#151
alexispurslane wants to merge 1 commit into
charmbracelet:mainfrom
alexispurslane:fix/zero-width-grapheme-loss

Conversation

@alexispurslane

@alexispurslane alexispurslane commented Aug 8, 2026

Copy link
Copy Markdown

Problem

printString (styled.go) drops zero-width graphemes when decomposing a styled string into cells. Width-0 sequences (ZWNJ U+200C, ZWSP U+200B, combining marks that decode as width-0) are accumulated into the pending cell in the default branch (cell.Content += string(seq)), but the next visible cell overwrites them:

case 1, 2, 3, 4:
    cell.Width = width
    cell.Content = string(seq)  // clobbers the accumulated zero-width graphemes

So any content containing a standalone width-0 grapheme loses it on the round-trip. A decomposed accent renders wrong: "e\u0301x" comes back as "ex".

Repro

ss := uv.NewStyledString("e\u0301x")
area := ss.Bounds()
buf := uv.NewScreenBuffer(area.Dx(), area.Dy())
ss.Draw(buf, area)
// buf.Lines[0] is [ {e}, {x} ] — the combining acute is gone

Fix

Prepend the accumulated zero-width content to the next visible cell instead of overwriting it. Input order is preserved (the accumulated sequences precede the visible grapheme in the input), cell widths are unchanged, and the end-of-string flush already emits a trailing width-0 cell, so trailing graphemes survive too.

Test

Two cases added to TestStyledString: ZWSP between two letters and a combining mark before a letter. Both fail on main and pass with the fix.

Context

Found while building a TUI editor: the canvas round-trip (lipgloss layer compositing) silently stripped ZWNJ and combining marks from editor content every frame. Note that #95 is adjacent (unknown escape sequences are dropped in the same default branch) but is a separate bug.

printString accumulates width-0 sequences (ZWNJ, ZWSP, combining marks
decoded as separate width-0 sequences) into the pending cell, but the
next visible cell overwrites them instead of prepending them, so the
graphemes are silently dropped from rendered output. A decomposed
accent like "e\u0301x" renders as "ex".

Prepend the accumulated zero-width content to the next visible cell.
Input order and cell widths are unchanged; only the dropped bytes are
recovered.

Fixes the loss observed when round-tripping content through a canvas
(e.g. lipgloss layer compositing).
@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.52%. Comparing base (19049f2) to head (c9372ef).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #151      +/-   ##
==========================================
+ Coverage   60.49%   60.52%   +0.02%     
==========================================
  Files          52       52              
  Lines        6751     6751              
==========================================
+ Hits         4084     4086       +2     
+ Misses       2391     2389       -2     
  Partials      276      276              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexispurslane

Copy link
Copy Markdown
Author

CI note: the conformance failures are pre-existing on main

The red conformance job is not caused by this change. I diffed the failing seeds:

Failing test main nightly (run 31243567629, 06:17 today) this PR (run 31256704853)
FuzzRenderer/8e096e0ae397d854 fails fails
FuzzScreenShowsContent/seed#28-31,34-37 fails fails
FuzzRedrawResyncs fails (new crasher from tonight's fuzz job) n/a (fuzz job is schedule-only)

PR-only failures: none. Every failing seed in this PR's run fails identically on main's own nightly test job, which ran hours before this PR existed. The committed crashers in internal/conformance/testdata/fuzz/ fail on every pull_request until the renderer bugs behind them are fixed — related territory to #140/#148/#150 (wide-cell handling across resizes).

The root-module CI (unit tests, go test ./... on main + this branch) is green; the new TestStyledString cases are red without the fix and green with it.

@alexispurslane

Copy link
Copy Markdown
Author

My agent commented the solution before even asking me, sorry about that! But yes, this appears to be a pre-existing issue.

@alexispurslane
alexispurslane marked this pull request as ready for review August 8, 2026 17:23
@taciturnaxolotl

Copy link
Copy Markdown
Member

if you rebase on main the conformance test should be clean now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants