Skip to content

fix(config): support Unicode keymap aliases - #1004

Open
tianrking wants to merge 1 commit into
rmk-rs:mainfrom
tianrking:codex/keymap-unicode-aliases
Open

fix(config): support Unicode keymap aliases#1004
tianrking wants to merge 1 commit into
rmk-rs:mainfrom
tianrking:codex/keymap-unicode-aliases

Conversation

@tianrking

@tianrking tianrking commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Related to #569.

Summary

  • Parse alias identifiers on UTF-8 character boundaries instead of byte offsets, preventing non-ASCII names from producing invalid string slices.
  • Use the same Unicode whitespace definition for alias scanning and keymap parsing, so separators such as NBSP and ideographic space remain valid after expansion.
  • Preserve the existing alias delimiters, case sensitivity, recursion limit, and [aliases] configuration format.
  • Add regression coverage for direct Unicode aliases, recursive expansion into an action, undefined Unicode aliases, and Unicode whitespace separators.
  • Document the supported behavior and add a changelog entry.

Validation

  • cargo test --manifest-path rmk-config/Cargo.toml (92 unit tests and 6 integration tests passed; doc tests passed)
  • cargo clippy --manifest-path rmk-config/Cargo.toml --all-targets --all-features -- -D warnings
  • cargo +nightly fmt --manifest-path rmk-config/Cargo.toml --check
  • git diff upstream/main...HEAD --check

Scope

This PR fixes Unicode handling in the existing keyboard.toml [aliases] path. It does not add the Rust-code alias API requested in #569, so that issue should remain open.

@github-actions

github-actions Bot commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Size Report

Example main PR Diff .text .data .bss
use_config/nrf52832_ble 398.1 KiB 398.1 KiB +0.00% ⬇️ -4 0 0
use_config/nrf52840_ble 440.7 KiB 440.7 KiB +0.00% ⬇️ -4 0 0
use_config/nrf52840_ble_split (central) 502.1 KiB 502.1 KiB +0.00% 0 0 0
use_config/nrf52840_ble_split (peripheral) 317.5 KiB 317.5 KiB +0.00% 0 0 0
use_config/pi_pico_w_ble 673.3 KiB 673.3 KiB +0.00% ⬇️ -4 0 0
use_config/rp2040 146.0 KiB 146.0 KiB +0.00% 0 0 0
use_config/rp2040_split (central) 159.6 KiB 159.6 KiB +0.00% ⬆️ +4 0 0
use_config/rp2040_split (peripheral) 27.9 KiB 27.9 KiB +0.00% 0 0 0
use_config/stm32f1 62.7 KiB 62.7 KiB +0.00% ⬆️ +4 0 0
use_config/stm32h7 100.8 KiB 100.8 KiB +0.00% ⬆️ +4 0 0
use_rust/nrf52832_ble 386.0 KiB 386.0 KiB +0.00% 0 0 0
use_rust/nrf52840_ble 435.6 KiB 435.6 KiB +0.00% ⬇️ -4 0 0
use_rust/nrf52840_ble_split (central) 505.6 KiB 505.6 KiB +0.00% ⬆️ +8 0 0
use_rust/nrf52840_ble_split (peripheral) 315.3 KiB 315.3 KiB +0.00% ⬇️ -4 0 0
use_rust/pi_pico_w_ble 672.7 KiB 672.7 KiB +0.00% ⬇️ -4 0 0
use_rust/rp2040 145.5 KiB 145.5 KiB +0.00% 0 0 0
use_rust/rp2040_split (central) 158.3 KiB 158.3 KiB +0.00% 0 0 0
use_rust/rp2040_split (peripheral) 28.2 KiB 28.2 KiB +0.00% 0 0 0
use_rust/stm32f1 61.9 KiB 61.9 KiB +0.00% 0 0 0
use_rust/stm32h7 119.6 KiB 119.6 KiB +0.00% 0 0 0
use_config/nrf52832_ble — 398.1 KiB → 398.1 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 365360	   8620	  33672	 407652	  63864	rmk-nrf52832

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 365364	   8620	  33672	 407656	  63868	rmk-nrf52832

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +12  [ = ]       0    .debug_str
   +10%      +5  [ = ]       0    [Unmapped]
  +0.0%      +3  [ = ]       0    .strtab
  -0.0%      -4  -0.0%      -4    .text
  -0.0%    -216  [ = ]       0    .debug_info
  -0.1%    -240  [ = ]       0    .debug_line
  -0.0%    -440  -0.0%      -4    TOTAL
use_config/nrf52840_ble — 440.7 KiB → 440.7 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 398948	   8628	  43704	 451280	  6e2d0	rmk-nrf52840

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 398952	   8628	  43704	 451284	  6e2d4	rmk-nrf52840

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.5%     +39  [ = ]       0    .debug_abbrev
  +0.0%     +12  [ = ]       0    .debug_str
  +0.0%     +10  [ = ]       0    .debug_loc
  +0.0%      +3  [ = ]       0    .strtab
  -3.6%      -2  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  -0.0%     -71  [ = ]       0    .debug_info
  -0.0%    -143  [ = ]       0    .debug_line
  -0.0%    -156  -0.0%      -4    TOTAL
use_config/nrf52840_ble_split (central) — 502.1 KiB → 502.1 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 456400	   8684	  49040	 514124	  7d84c	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 456400	   8684	  49040	 514124	  7d84c	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +12  [ = ]       0    .debug_str
  +0.0%      +3  [ = ]       0    .strtab
  -2.0%      -1  [ = ]       0    [Unmapped]
  -0.0%     -10  [ = ]       0    .debug_loc
  -0.0%    -216  [ = ]       0    .debug_info
  -0.1%    -240  [ = ]       0    .debug_line
  -0.0%    -452  [ = ]       0    TOTAL
use_config/nrf52840_ble_split (peripheral) — 317.5 KiB → 317.5 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 291464	   8500	  25184	 325148	  4f61c	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 291464	   8500	  25184	 325148	  4f61c	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  -3.5%      -2  [ = ]       0    [Unmapped]
  -0.0%     -10  [ = ]       0    .debug_loc
  -0.0%     -12  [ = ]       0    TOTAL
use_config/pi_pico_w_ble — 673.3 KiB → 673.3 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 635688	      0	  53736	 689424	  a8510	rmk-pi-pico-w

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 635692	      0	  53736	 689428	  a8514	rmk-pi-pico-w

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +465  [ = ]       0    .debug_info
  +0.0%    +165  [ = ]       0    .debug_str
  +0.0%    +125  [ = ]       0    .debug_line
  +0.4%     +39  [ = ]       0    .debug_abbrev
  +0.0%     +23  [ = ]       0    .strtab
  -4.2%      -3  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  -0.0%     -10  [ = ]       0    .debug_loc
  +0.0%    +800  -0.0%      -4    TOTAL
use_config/rp2040 — 146.0 KiB → 146.0 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 135656	      0	  13884	 149540	  24824	rmk-rp2040

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 135656	      0	  13884	 149540	  24824	rmk-rp2040

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +72  [ = ]       0    .debug_str
  +0.0%     +43  [ = ]       0    .debug_info
  +0.0%     +10  [ = ]       0    .debug_loc
  +0.0%      +8  [ = ]       0    .strtab
  -7.3%      -3  [ = ]       0    [Unmapped]
  -0.0%     -58  [ = ]       0    .debug_line
  +0.0%     +72  [ = ]       0    TOTAL
use_config/rp2040_split (central) — 159.6 KiB → 159.6 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 148628	      0	  14836	 163464	  27e88	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 148624	      0	  14836	 163460	  27e84	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%      +4  +0.0%      +4    .text
  -1.6%      -1  [ = ]       0    [Unmapped]
  -0.0%      -5  [ = ]       0    .strtab
  -0.0%     -37  [ = ]       0    .debug_info
  -0.0%    -105  [ = ]       0    .debug_str
  -0.1%    -124  [ = ]       0    .debug_line
  -0.0%    -268  +0.0%      +4    TOTAL
use_config/rp2040_split (peripheral) — 27.9 KiB → 27.9 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  25696	     60	   2804	  28560	   6f90	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  25696	     60	   2804	  28560	   6f90	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL
use_config/stm32f1 — 62.7 KiB → 62.7 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  57812	     28	   6344	  64184	   fab8	rmk-stm32f1

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  57808	     28	   6344	  64180	   fab4	rmk-stm32f1

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%      +4  +0.0%      +4    .text
  -4.5%      -3  [ = ]       0    [Unmapped]
  -0.0%      -9  [ = ]       0    .strtab
  -0.0%     -52  [ = ]       0    .debug_info
  -0.0%     -64  [ = ]       0    .debug_str
  -0.0%    -124  +0.0%      +4    TOTAL
use_config/stm32h7 — 100.8 KiB → 100.8 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  94804	    264	   8104	 103172	  19304	rmk-stm32h7

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  94800	    264	   8104	 103168	  19300	rmk-stm32h7

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1%    +124  [ = ]       0    .debug_line
  +0.0%     +37  [ = ]       0    .debug_str
  +0.0%      +4  +0.0%      +4    .text
 -13.8%      -8  [ = ]       0    [Unmapped]
  -0.0%      -9  [ = ]       0    .strtab
  -0.0%     -16  [ = ]       0    .debug_info
  +0.0%    +132  +0.0%      +4    TOTAL
use_rust/nrf52832_ble — 386.0 KiB → 386.0 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 352992	   8620	  33608	 395220	  607d4	rmk-nrf52832

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 352992	   8620	  33608	 395220	  607d4	rmk-nrf52832

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1% +2.27Ki  [ = ]       0    .debug_str
  +0.1%    +341  [ = ]       0    .strtab
  -1.5%      -1  [ = ]       0    [Unmapped]
  -0.0%     -10  [ = ]       0    .debug_line
  -0.0%     -10  [ = ]       0    .debug_loc
  -0.0%     -72  [ = ]       0    .debug_info
  +0.0% +2.51Ki  [ = ]       0    TOTAL
use_rust/nrf52840_ble — 435.6 KiB → 435.6 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 396460	   8628	  40944	 446032	  6ce50	rmk-nrf52840

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 396464	   8628	  40944	 446036	  6ce54	rmk-nrf52840

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +30  [ = ]       0    .strtab
  -2.1%      -1  [ = ]       0    [Unmapped]
  -0.0%      -3  [ = ]       0    .debug_line
  -0.0%      -4  -0.0%      -4    .text
  -0.0%     -10  [ = ]       0    .debug_loc
  -0.0%     -40  [ = ]       0    .debug_str
  -0.0%     -72  [ = ]       0    .debug_info
  -0.0%    -100  -0.0%      -4    TOTAL
use_rust/nrf52840_ble_split (central) — 505.6 KiB → 505.6 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 458064	   8684	  51008	 517756	  7e67c	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 458056	   8684	  51008	 517748	  7e674	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +30  [ = ]       0    .strtab
  +0.0%      +8  +0.0%      +8    .text
 -10.0%      -6  [ = ]       0    [Unmapped]
  -0.2%     -19  [ = ]       0    .debug_abbrev
  -0.0%     -91  [ = ]       0    .debug_line
  -0.0%    -101  [ = ]       0    .debug_str
  -0.0%    -249  [ = ]       0    .debug_info
  -0.0%    -428  +0.0%      +8    TOTAL
use_rust/nrf52840_ble_split (peripheral) — 315.3 KiB → 315.3 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 289448	   8500	  24952	 322900	  4ed54	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 289452	   8500	  24952	 322904	  4ed58	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +232  [ = ]       0    .debug_info
  +0.0%    +155  [ = ]       0    .debug_str
  +0.0%     +98  [ = ]       0    .debug_line
  +0.0%      +4  [ = ]       0    .strtab
  -1.9%      -1  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  +0.0%    +484  -0.0%      -4    TOTAL
use_rust/pi_pico_w_ble — 672.7 KiB → 672.7 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 635464	      0	  53416	 688880	  a82f0	rmk-pi-pico-w

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 635468	      0	  53416	 688884	  a82f4	rmk-pi-pico-w

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +4.4%      +3  [ = ]       0    [Unmapped]
  +0.0%      +1  [ = ]       0    .debug_line
  -0.0%      -4  -0.0%      -4    .text
  [ = ]       0  -0.0%      -4    TOTAL
use_rust/rp2040 — 145.5 KiB → 145.5 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 135276	      0	  13724	 149000	  24608	rmk-rp2040

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 135276	      0	  13724	 149000	  24608	rmk-rp2040

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +101  [ = ]       0    .debug_str
  +0.0%     +80  [ = ]       0    .debug_line
  +0.0%     +37  [ = ]       0    .debug_info
  +8.8%      +3  [ = ]       0    [Unmapped]
  +0.0%      +1  [ = ]       0    .strtab
  -0.0%     -10  [ = ]       0    .debug_loc
  +0.0%    +212  [ = ]       0    TOTAL
use_rust/rp2040_split (central) — 158.3 KiB → 158.3 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 147560	      0	  14580	 162140	  2795c	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 147560	      0	  14580	 162140	  2795c	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1% +2.36Ki  [ = ]       0    .debug_str
  +0.2%    +242  [ = ]       0    .strtab
  +7.9%      +3  [ = ]       0    [Unmapped]
  -0.0%     -77  [ = ]       0    .debug_info
  -0.1%    -105  [ = ]       0    .debug_line
  +0.1% +2.42Ki  [ = ]       0    TOTAL
use_rust/rp2040_split (peripheral) — 28.2 KiB → 28.2 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  25796	     60	   3068	  28924	   70fc	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  25796	     60	   3068	  28924	   70fc	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL
use_rust/stm32f1 — 61.9 KiB → 61.9 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  57128	     28	   6272	  63428	   f7c4	rmk-stm32f1

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  57128	     28	   6272	  63428	   f7c4	rmk-stm32f1

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  -0.0%      -1  [ = ]       0    .debug_str
  -6.4%      -3  [ = ]       0    [Unmapped]
  -0.0%      -4  [ = ]       0    TOTAL
use_rust/stm32h7 — 119.6 KiB → 119.6 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 108148	    320	  14004	 122472	  1de68	rmk-stm32h7

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 108148	    320	  14004	 122472	  1de68	rmk-stm32h7

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +10  [ = ]       0    .debug_loc
  +1.7%      +1  [ = ]       0    [Unmapped]
  -0.0%      -1  [ = ]       0    .strtab
  -0.0%     -37  [ = ]       0    .debug_info
  -0.0%    -101  [ = ]       0    .debug_str
  -0.1%    -124  [ = ]       0    .debug_line
  -0.0%    -252  [ = ]       0    TOTAL

@tianrking
tianrking marked this pull request as ready for review July 26, 2026 17:31
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 26, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46d66a2247

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rmk-config/src/layout.rs Outdated
Comment on lines +267 to +269
let alias_len = current_keys[alias_start..]
.find(char::is_whitespace)
.unwrap_or(current_keys.len() - alias_start);

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make Unicode alias separators valid keymap whitespace

When an alias is followed by non-ASCII whitespace such as NBSP or an ideographic space, this scan ends the alias at that character but leaves the separator in next_keys. The unchanged keymap.pest WHITESPACE rule accepts only space, tab, newline, and carriage return, so the Unicode separator is absorbed into an adjacent loose_identifier or causes parsing to fail, producing an invalid action or incorrect key count. Normalize/consume these separators during expansion or extend the grammar to recognize the same whitespace set.

Useful? React with 👍 / 👎.

@HaoboGu

HaoboGu commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

This cannot close #569

@HaoboGu

HaoboGu commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@tianrking Hey could you please update the PR and fix the issue/comment? Thanks!

@tianrking
tianrking force-pushed the codex/keymap-unicode-aliases branch from 46d66a2 to 94cc07d Compare August 12, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tianrking @HaoboGu