perf: fast native UTF-16 source copy into Wasm memory on Node#224
Merged
Conversation
guybedford
force-pushed
the
perf/node-utf16-copy
branch
from
July 8, 2026 00:15
c48ccc1 to
cf4df7c
Compare
guybedford
force-pushed
the
perf/node-utf16-copy
branch
from
July 8, 2026 00:16
cf4df7c to
db32979
Compare
mergify Bot
added a commit
to ArcadeData/arcadedb
that referenced
this pull request
Jul 12, 2026
…2.3.1 in /studio [skip ci] Bumps [es-module-lexer](https://github.com/guybedford/es-module-lexer) from 2.3.0 to 2.3.1. Release notes *Sourced from [es-module-lexer's releases](https://github.com/guybedford/es-module-lexer/releases).* > 2.3.1 > ----- > > What's Changed > -------------- > > * perf: fast native UTF-16 source copy into Wasm memory on Node by [`@guybedford`](https://github.com/guybedford) in [guybedford/es-module-lexer#224](https://redirect.github.com/guybedford/es-module-lexer/pull/224) > * perf(lexer): gate keywordStart on the next byte for i/c tokens by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#223](https://redirect.github.com/guybedford/es-module-lexer/pull/223) > > **Full Changelog**: <guybedford/es-module-lexer@2.3.0...2.3.1> Commits * [`2b2e620`](guybedford/es-module-lexer@2b2e620) 2.3.1 * [`8c21b74`](guybedford/es-module-lexer@8c21b74) perf(lexer): gate keywordStart on the next byte for i/c tokens ([#223](https://redirect.github.com/guybedford/es-module-lexer/issues/223)) * [`b8f9d89`](guybedford/es-module-lexer@b8f9d89) perf: fast native UTF-16 source copy into Wasm memory on Node ([#224](https://redirect.github.com/guybedford/es-module-lexer/issues/224)) * See full diff in [compare view](guybedford/es-module-lexer@2.3.0...2.3.1) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copying the JS source into Wasm linear memory is ~25% of total parse time, and it was done with a per-character
charCodeAtloop (~1.2 GB/s). On Node we can instead encode the string as UTF-16 straight into memory natively viaBuffer(~15 GB/s, ~13x), with no other lexer changes since the lexer already indexes UTF-16 code units.Warm parse of the full sample corpus (3057 KiB, avg of 25 runs):
The whole gain comes from a single capability: encoding a JS string as UTF-16 directly into linear memory. Node exposes it natively; the browser has no equivalent (
TextEncoderis UTF-8 only), so the portablecharCodeAtpath is retained as a fallback and the same speedup is left on the table anywhere a native UTF-16 encode-into-memory becomes available.The fast path is gated strictly to the real Node
Bufferon little-endian hosts (every platform Node runs on); browser bundles and big-endian hosts keep the existing code-unit copy. Existing tests pass unchanged, including the non-ASCII sample files.