perf(lexer): gate keywordStart on the next byte for i/c tokens#223
Merged
guybedford merged 3 commits intoJul 12, 2026
Merged
Conversation
The main character loop calls keywordStart() (and through it a second boundary helper) on every `i` and `c` it reaches, but over real-world sources the next byte rules the token out ~98% of the time for `i` and ~95% for `c`. Checking `*(pos + 1)` against the keyword's second letter first short-circuits those calls with a single load+compare; memcmp already requires that same byte, so the guard changes nothing observable. Measured on the angular/d3/rollup/magic-string corpus (~3M code units, Node 24 / V8, n=150 iters x 13 trials, drop best+worst): -1.4% mean parse time, parse output byte-identical. Wasm shrinks 229 bytes / 129 instructions (the drive-by below covers most of the byte delta). Drive-by fix: * isBrOrWs: write 9..13 as `(char16_t)(c - 9) < 5`, one unsigned compare instead of two, shrinking every inlined copy of the helper.
BridgeAR
marked this pull request as ready for review
July 8, 2026 09:02
guybedford
reviewed
Jul 12, 2026
guybedford
left a comment
Owner
There was a problem hiding this comment.
Looks good thanks, although seems wasteful to do the double compare if we can avoid it?
The pos[1]=='m' gate already fixes the first byte, so the follow-up memcmp only needs to check "port". Replace MPORT with PORT and start the compare at pos+2, dropping the redundant re-read of pos[1] (-6 wasm instructions).
Same as the import gate: pos[1]=='l' already fixes the first byte, so the memcmp only needs "ass". Slice &LASS[1] (LASS stays for the two non-fast-path class checks). -4 bytes, no extra instructions.
guybedford
enabled auto-merge (squash)
July 12, 2026 02:03
mergify Bot
added a commit
to ArcadeData/arcadedb
that referenced
this pull request
Jul 12, 2026
…2.3.1 in /studio [skip ci] Bumps [es-module-lexer](https://github.com/guybedford/es-module-lexer) from 2.3.0 to 2.3.1. Release notes *Sourced from [es-module-lexer's releases](https://github.com/guybedford/es-module-lexer/releases).* > 2.3.1 > ----- > > What's Changed > -------------- > > * perf: fast native UTF-16 source copy into Wasm memory on Node by [`@guybedford`](https://github.com/guybedford) in [guybedford/es-module-lexer#224](https://redirect.github.com/guybedford/es-module-lexer/pull/224) > * perf(lexer): gate keywordStart on the next byte for i/c tokens by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#223](https://redirect.github.com/guybedford/es-module-lexer/pull/223) > > **Full Changelog**: <guybedford/es-module-lexer@2.3.0...2.3.1> Commits * [`2b2e620`](guybedford/es-module-lexer@2b2e620) 2.3.1 * [`8c21b74`](guybedford/es-module-lexer@8c21b74) perf(lexer): gate keywordStart on the next byte for i/c tokens ([#223](https://redirect.github.com/guybedford/es-module-lexer/issues/223)) * [`b8f9d89`](guybedford/es-module-lexer@b8f9d89) perf: fast native UTF-16 source copy into Wasm memory on Node ([#224](https://redirect.github.com/guybedford/es-module-lexer/issues/224)) * See full diff in [compare view](guybedford/es-module-lexer@2.3.0...2.3.1) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The main character loop calls
keywordStart()(and through it a second boundary helper) on everyiandctoken it reaches. On the angular/d3/rollup/magic-string corpus the byte after the token rules it out ~98% of the time foriand ~95% forc, so those calls are almost always wasted. Checking*(pos + 1)against the keyword's second letter (mforimport,lforclass) first short-circuits the calls with a single load+compare.memcmpalready requires that same byte, so the guard is behaviour-preserving.Why
The per-character loop is the dominant cost (it runs once per source code unit; the boundary helpers are called <0.05x per code unit), so cutting a function call out of the two hottest keyword cases moves the needle where micro-tuning the helpers themselves does not.
Measured on the same corpus the README benchmarks (~3M code units, Node 24 / V8, 150 iters x 13 trials, drop best+worst, run against the built wasm):
Drive-by
isBrOrWs: write the9..13range as(char16_t)(c - 9) < 5, one unsigned compare instead of two. The wasm backend keeps the shorter form at every inlined copy of the helper; this accounts for most of the byte/instruction reduction above.