Skip to content

Read the keyboard from the X keymap, not from SDL's private X11_KeyToUnicode - #90

Merged
greenfire27 merged 1 commit into
developmentfrom
linux-x11-keytounicode
Sep 3, 2026
Merged

Read the keyboard from the X keymap, not from SDL's private X11_KeyToUnicode#90
greenfire27 merged 1 commit into
developmentfrom
linux-x11-keytounicode

Conversation

@greenfire27

Copy link
Copy Markdown
Collaborator

The problem

x86UNIXInputManager.cc declared extern "C" Uint16 X11_KeyToUnicode(SDLKey, SDLMod) and called it three times per key to fill AsciiTable with the unshifted, shifted and AltGr character for every key. That function belongs to SDL 1.2's X11 driver, appears in no public header, and linked only because SDL 1.2 was built without symbol visibility control.

Distributions no longer ship that SDL — what they package as SDL 1.2 is sdl12-compat, a shim over SDL2/3. The interesting part is that it does not fail to link: it exports an X11_KeyToUnicode of its own, and that one is a US-layout toupper() stub. Measured against a hand-built genuine SDL 1.2, same machine, same us layout:

keystroke genuine SDL 1.2 sdl12-compat
shift-1 ! 1
shift-; : ;
shift-/ ? /
shift-- _ -
AltGr row layout's AltGr character copy of the unshifted row

So building against a distribution package linked cleanly and then mistyped every shifted punctuation character — a worse failure than a missing symbol, because nothing announces it.

The change

The ascii table is now read from the X keymap through public Xlib, which is where SDL was reading it from anyway. SDLKeyToXKeySym names the X keysym for each SDL keysym, and XLookupString against a synthetic XKeyEvent asks the server what that key types with no modifier, with ShiftMask, and with Mod5Mask for AltGr. The display comes from the existing DisplayPtrManager, so it also works when SDL has not stored one, and a missing display now warns instead of silently zeroing the table.

Deliberately not the approach the Emscripten back-end takes (a hardcoded printable-ASCII filter plus a US-QWERTY shift switch) — that is right for a browser canvas and would regress every non-US keyboard on desktop Linux.

A trap worth recording

The keypad entries name the numlock-off keysyms (XK_KP_Delete, XK_KP_Insert, …) rather than XK_KP_Decimal and XK_KP_0. A keyboard binds both spellings and XKeysymToKeycode resolves them to different keycodes — XK_KP_Decimal landed on a separate layout-defined separator key (129) instead of the physical numpad period (91). The numlock-off spelling is what SDL 1.2 effectively resolved to, and this was the single mismatch until it was fixed.

A second, unrelated wall

Fixing the link exposed one. InitSDL takes the engine's Display, screen and locking out of SDL_GetWMInfo's x11 union, and sdl12-compat picks the Wayland video driver on a Wayland desktop, where that call answers "No SysWM support available". The engine stopped at "Unable to initialize SDL" before it ever reached a keymap. It now asks for SDL's x11 driver with a non-overriding setenv and runs through Xwayland; an explicit SDL_VIDEODRIVER still wins.

Verification

Ran the identical translation code beside genuine SDL 1.2 over all 113 keys InitKeyMaps maps:

  • us layout — 0 differences.
  • de layout — unshifted and shifted rows also identical; the AltGr row now holds what AltGr actually produces (AltGr-q is @) instead of repeating the unshifted character. That is the one intended behaviour change, and it is what the "goofy (i18n) case" exists for.

On Arch against sdl12-compat 1.2.68, with no vendored SDL in the configure:

  • readelf -d reports no RUNPATH at all (previously a path into a gitignored build/deps/sdl12, which git clean -xdf would have broken).
  • nm -u shows X11_KeyToUnicode is not referenced.
  • The editor boots, loads PlanetX, and typing ! : ? _ " ( into the in-engine console produces exactly those characters — where the stub would have produced 1 ; / - ' 9.

An A/B against a pre-change build on genuine SDL 1.2, under identical synthetic input, was indistinguishable.

Also here

  • Deleted the vestigial X11_KeyToUnicode extern in EmscriptenInputManager.cpp, which nothing called.
  • Dropped the ubuntu-22.04 pin on the linux-x64 CI job, which existed only for this. This is the one thing only CI can confirm — 24.04's libsdl1.2-dev resolves through a transitional package name, which I could not verify locally. The 32-bit job keeps its pin, but re-labelled for the :i386 dev packages, which is the reason that still actually applies.
  • CMakeLists.txt, cmake/BUILD-PLATFORM-NOTES.md and build-linux.sh all claimed the shim lacks the symbol. Measurement says that is not what goes wrong, so they now say what does.
  • Two CHANGELOG.md entries under the unreleased section.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CnwJvxtXJ6Vb1BHEVgUm6o

Every key's character came from X11_KeyToUnicode: a function of SDL 1.2's X11
driver that appears in no public header, declared in x86UNIXInputManager.cc as
a bare extern and called three times per key to fill AsciiTable with the
unshifted, shifted and AltGr character. It linked only because SDL 1.2 was
built without symbol visibility control.

Distributions no longer ship that SDL. What they call SDL 1.2 is sdl12-compat,
a shim over SDL2/3, and the interesting part is that it does not fail to link:
it exports an X11_KeyToUnicode of its own. That one is a US-layout toupper()
stub. Measured against a hand-built genuine SDL 1.2 on the same machine and
the same us layout, shift-1 answered "1" instead of "!", shift-; answered ";"
instead of ":", shift-/ answered "/" instead of "?", shift-- answered "-"
instead of "_", and the AltGr row was a copy of the unshifted one. So building
against a distribution package linked cleanly and then mistyped every shifted
punctuation character, which is a worse failure than a missing symbol.

The table is now read from the X keymap through public Xlib, which is where
SDL was reading it from: SDLKeyToXKeySym names the X keysym for each SDL one,
and XLookupString against a synthetic XKeyEvent asks the server what that key
types with no modifier, with Shift, and with Mod5 for AltGr. The display comes
from the existing DisplayPtrManager, so it also works when SDL has not stored
one, and a missing display now says so instead of silently zeroing the table.

Checked by running the identical translation code beside genuine SDL 1.2 over
all 113 keys InitKeyMaps maps: no differences on a us layout. On a de layout
the unshifted and shifted rows are also identical, and the AltGr row now holds
what AltGr actually produces (AltGr-q is '@') rather than repeating the
unshifted character -- the "goofy (i18n) case" finally being the i18n case.

One trap worth recording. The keypad entries name the numlock-OFF keysyms
(XK_KP_Delete, XK_KP_Insert and friends) rather than XK_KP_Decimal and
XK_KP_0. A keyboard binds both spellings, and XKeysymToKeycode resolves them
to different keycodes: XK_KP_Decimal landed on a separate layout-defined
separator key (129) instead of the physical numpad period (91). Naming the
numlock-off spelling is what SDL 1.2 effectively did, and it was the single
mismatch until it was fixed.

Fixing the link exposed a second, unrelated wall. InitSDL takes the engine's
Display, screen and locking out of SDL_GetWMInfo's x11 union, and sdl12-compat
picks the Wayland video driver on a Wayland desktop, where that call answers
"No SysWM support available" -- so the engine stopped at "Unable to initialize
SDL" before it ever got to a keymap. It now asks for SDL's x11 driver with a
non-overriding setenv and runs through Xwayland; an explicit SDL_VIDEODRIVER
still wins.

Verified on Arch against sdl12-compat 1.2.68 with no vendored SDL in the
configure: readelf -d reports no RUNPATH at all, nm -u shows X11_KeyToUnicode
is not referenced, the editor boots and loads PlanetX, and typing ! : ? _ " (
into the in-engine console produces exactly those characters where the stub
would have produced 1 ; / - ' 9.

Also here: EmscriptenInputManager.cpp carried a vestigial extern for the same
symbol that nothing called. The linux-x64 CI job loses its ubuntu-22.04 pin,
which existed only for this; the 32-bit job keeps its pin, but for the :i386
dev packages, which is the reason that actually still applies. CMakeLists,
BUILD-PLATFORM-NOTES and build-linux.sh all claimed the shim lacks the symbol,
which measurement says is not what goes wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnwJvxtXJ6Vb1BHEVgUm6o
@greenfire27
greenfire27 merged commit 91e4221 into development Sep 3, 2026
18 checks passed
@greenfire27
greenfire27 deleted the linux-x11-keytounicode branch September 3, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant