Skip to content

Implement zcr_extended_text_input_v1 handlers and fix IME backspace holding - #27

Closed
kkimdev wants to merge 40 commits into
google:virtwlfrom
kkimdev:feature/wayland-ime
Closed

Implement zcr_extended_text_input_v1 handlers and fix IME backspace holding#27
kkimdev wants to merge 40 commits into
google:virtwlfrom
kkimdev:feature/wayland-ime

Conversation

@kkimdev

@kkimdev kkimdev commented Jul 4, 2026

Copy link
Copy Markdown

Reproducible bugs fixed:

  1. Hold Backspace during Korean IME composition → only the last syllable is deleted. Type "가나다라마" and hold Backspace. "마" (the preedit) disappears, but "가나다라" stays and the repeated delete never happens. Root cause: v3 has no event for "delete one character on repeat". This PR adds a heuristic (preedit_cleared_for_backspace flag) that detects backspace bursts and synthesizes wl_keyboard.key events to bridge the gap.

  2. Type "가나다" and press Enter → only "가나" stays, "다" disappears. Root cause: zcr_extended_text_input_v1::set_preedit_region was a no-op. ChromeOS IME calls this on every keystroke to position the preedit within surrounding text. Without it, the cursor is wrong when committing, so the last syllable writes to the wrong offset.

Other issues fixed:

  1. zcr_extended_text_input_v1::confirm_preedit was a no-op. Confirming preedit silently did nothing. Now translates into v3 preedit_string("") + commit_string + done.

  2. done_serial always 0. on_commit_string sent hardcoded 0 for the v3 done serial. Guests that validate serials reject updates.

  3. commit_state always sent serial 0. host_serial from v1 IME events was never propagated to the host's commit_state.

  4. Activation tied only to enable/disable commit. Keyboard enter/leave had no effect — IME could appear active on an unfocused surface or inactive on a focused one. Replaced with update_host_activation state machine.

  5. surrounding_text consumed on every commit. The old code used .take(), losing surrounding text after one forward.

  6. Missing preedit_string("") before commit_string. Without it, some guests keep a stale composition underline after committing.

Implementation: replaced manual wire-format building (~150 lines of extend_from_slice and len<<16) with MessageBuilder + push_msg helper, applied early-exit pattern, added 48 tests.

Testing:

cargo build --release
(trap 'kill 0' INT; \
  cargo run --release -- wayland-test-1 & \
  sleep 5; \
  WAYLAND_DISPLAY=wayland-test-1 code)

kkimdev added 30 commits July 4, 2026 06:52
…ion and confirm_preedit

Map transactional host CJK/Korean IME preedit region changes and confirmation events to v3 delete_surrounding_text, preedit_string, commit_string, and done event sequences.
Track the host's event serials on preedit_string, commit_string, keysym, language, and text_direction, and propagate them in commit_state requests. This ensures bidirectional serialization sync and prevents the host compositor from ignoring text state updates, resolving the IME backspace ghosting/lock bug.
Pass the host-sent serial and time in wl_keyboard::key events generated from on_keysym instead of hardcoding them to 0. This ensures the guest client associates the synthetic key event with the correct user input transaction, fixing blocked/ignored backspace composition clearing bugs.
Track host activation state in TextInputState to avoid null-surface
activation crashes. Bind update_host_activation to focus transitions
and commit events. Forward host serials through commit_state.
Implement CJK backspace holding test.
…_msg

queue_host_msg/queue_client_msg took &mut Context, which prevented
Rust's field-level borrow splitting. This forced the use of 3 helper
structs (ActivationAction, CommitActions, PreeditAction) and the
extract-then-send pattern throughout the IME bridge.

Replace with push_msg(queue, sender, opcode, builder) that takes
the specific queue reference. This allows all handlers to queue
messages directly inside their get_mut/iter_mut blocks.

Removed:
- ActivationAction struct (update_host_activation)
- CommitActions struct (on_commit)
- PreeditAction struct (on_set_preedit_region)
- Tuple extract pattern (on_confirm_preedit)

Added: use std::os::unix::io::RawFd
Remove .direnv/, _local/, pr_body.local.md, test_e2e.local.ts
from tracking — these were accidentally committed local artifacts.

Apply rustfmt to changed source files.
… fix and logging

- Removed commit-parameter → commit_string forwarding from on_preedit_string
- Removed pending_implicit_commit field and all its uses
- Reverted to original behavior where commit parameter is ignored
- Kept done_serial as monotonic independent counter
- Kept comprehensive log::info! traces for debugging
- Remove commit_serial (field was incremented but never read)
- Remove thread_local XKB keymap caching; rebuild keymap per keysym event
  (keysym events are human-paced, caching adds unnecessary complexity)
- Keep all functional fixes: done_serial, host_serial, delete_surrounding_text,
  activation state machine, keysym serial/time forwarding
The v1→v3 conversion loses the commit parameter from preedit_string.
Korean IMEs rely on v1's implicit commit mechanism, which has no v3
equivalent and cannot be replicated without language-specific heuristics.
Logging added to all handlers:
- on_keysym: sym char, guest_id, keycode lookup result, forwarding
- on_delete_surrounding_text: index/length, before/after conversion
- on_set_preedit_region: host_ext_id, index/length
- on_confirm_preedit: host_ext_id, cached preedit text
- on_language / on_text_direction: serial and values
- v3 handlers: set_surrounding_text, set_content_type, set_cursor_rectangle,
  on_commit (enabled state, host_v1_id, forwarded ops, commit_state serial)
- wl_keyboard.on_enter/on_leave: keyboard/surface IDs, seat info, which
  text_inputs get updated
Logging now covers every event handler in both text_input.rs and keyboard.rs:
- on_modifiers_map, on_input_panel_state, on_preedit_styling,
  on_preedit_cursor, on_cursor_position (v1)
- on_clear_grammar_fragments, on_add_grammar_fragment,
  on_set_autocorrect_range, on_set_virtual_keyboard_occluded_bounds (ext)
- on_get_text_input (v3)
- on_modifiers, on_release (keyboard)
…_text

When the user holds backspace after clearing composition, the IME sends
repeated confirm_preedit (ChromeOS extension) events with empty preedit
instead of delete_surrounding_text. Our handler was treating these as
no-ops, making backspace stop after the preedit is cleared.

Now, when cached preedit is empty, confirm_preedit is forwarded as
v3 delete_surrounding_text(before=1, after=0) + done, deleting one
character per event.
When the IME pulls the last committed character into preedit and deletes
it component by component, continued backspace (empty preedit) generates
confirm_preedit events. The old fix used before_length=1 unconditionally,
which is wrong for Korean UTF-8 (3 bytes per character).

Track UTF-8 byte sizes of each committed character in a stack.
On confirm_preedit(empty), pop the last tracked size and use it as
before_length in delete_surrounding_text. Reset when the guest sends
set_surrounding_text.

This correctly handles mixed English+Korean text since each character's
actual byte length is recorded.
When backspace clears the preedit (non-empty to empty transition tracked
via preedit_cleared_for_backspace flag), subsequent confirm_preedit events
are the IME asking us to delete committed text. Instead of unreliable
delete_surrounding_text (wrong byte counts, guest may ignore it), synthesize
wl_keyboard.on_key(KEY_BACKSPACE=14) press+release to the guest keyboard.

This works universally: correct byte count for all encodings, works with
pre-existing text, and the guest natively handles backspace.

During initialization (no preedit clear) confirm_preedit is treated as a
no-op (just done), preventing spurious deletions.
Previous approach consumed the flag on first confirm_preedit, so only
one KEY_BACKSPACE was synthesized per backspace burst. The IME sends
one confirm_preedit per character to delete during hold; we need to
synthesize KEY_BACKSPACE for each one.

Make the flag sticky: set when preedit is cleared via backspace, and
only clear when:
- A new non-empty preedit starts (new composition)
- on_enter/on_leave (v1 input focus change)
- on_enable/on_disable (v3 input activation change)
- on_commit_string (commit normalizes state)

This way the entire burst of confirm_preedit events during backspace
hold correctly synthesizes one KEY_BACKSPACE per event.
The on_commit_string handler sends preedit_string('') + commit_string +
done to properly clear preedit on the guest. The confirm_preedit handler
(non-empty branch) was only sending commit_string + done — missing the
preedit_string('') clear. This caused stale IME tooltips on Enter because
the guest never received an explicit preedit clear before commit_string.

On Space, the IME uses commit_string (v1) which goes through the correct
handler. On Enter, the IME uses confirm_preedit (extension) which goes
through the now-fixed handler.
kkimdev added 10 commits July 4, 2026 20:23
This stack was maintained by commit_string and cleared by
set_surrounding_text, but never consumed — it was dead code from a
prior approach (delete_surrounding_text with byte tracking). All
backspace handling now uses KEY_BACKSPACE synthesis on the wl_keyboard
path, which doesn't need byte-level tracking.
Replace nested if-let blocks with let-else early-exit pattern in
on_commit_string, on_delete_surrounding_text, on_set_preedit_region,
on_confirm_preedit, on_commit (v3), on_enter, and on_leave.
Reduces indentation by one level in each function without changing behavior.
… trace

- Extract store_host_serial() helper for the 3 repeated nested if-let
  patterns in on_keysym, on_language, on_text_direction.
- Simplify on_commit trace logging to a single ctx.text_inputs.get().
- Downgrade on_modifiers from info! to trace! (fires per keystroke).
@kkimdev

kkimdev commented Jul 8, 2026

Copy link
Copy Markdown
Author

I've split this PR into a stacked series for easier review:

  1. refactor(ime): replace manual wire assembly with MessageBuilder/push_msg and add structured logging #28refactor(ime): replace manual wire assembly with MessageBuilder/push_msg and add structured loggingsubmitted
    Behavior-neutral. Replaces the ~150 lines of hand-rolled wire format with push_msg calls (byte-identical output). Adds structured logging to every handler. No protocol changes.

  2. PR 2 (coming soon) — Core IME correctness: set_preedit_region/confirm_preedit handlers, serial propagation, commit_state fix, activation state machine, delete_surrounding_text conversion, preedit_string("") fix.

  3. PR 3 (coming soon) — Korean IME backspace hold heuristic (the riskiest part, isolated as its own PR).

The stack builds on #28, so each diff is much smaller and focused.

@kkimdev

kkimdev commented Jul 9, 2026

Copy link
Copy Markdown
Author

Superseded by stacked PRs: #28 (refactor, merged) + #30 (IME fixes)

@kkimdev kkimdev closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant