⚡ perf: Optimize simhash bit generation and assembly#83
Closed
SuarezPM wants to merge 4 commits into
Closed
Conversation
Replaced inefficient nested Python loops for token bit generation and extraction in `_simhash_impl` with vectorized NumPy operations and `np.packbits`. Benchmarks indicate an ~12.5x speedup locally (3.14s -> 0.25s). Correctness is preserved by casting bit counts to `np.int32` before summing to properly handle negatives during the boolean vote. Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
Contributor
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
💡 **What:** Replaced the highly nested Python loops inside `_simhash_impl` with vectorized NumPy operations and fast bit-packing using `np.packbits` and `int.from_bytes()`. 🎯 **Why:** The existing implementation iterated bitwise 64 times for every token purely in Python, causing enormous performance degradation during scaling or long sequences. 📊 **Measured Improvement:** Utilizing an internal isolated benchmark with 1000 random token ids over 100 loops, execution time decreased from ~3.14s to ~0.25s, achieving a ~12.5x speedup while preserving bit-exact matching. Signed-off-by: Jules <jules@example.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
💡 **What:** Replaced the highly nested Python loops inside `_simhash_impl` with vectorized NumPy operations and fast bit-packing using `np.packbits` and `int.from_bytes()`. 🎯 **Why:** The existing implementation iterated bitwise 64 times for every token purely in Python, causing enormous performance degradation during scaling or long sequences. 📊 **Measured Improvement:** Utilizing an internal isolated benchmark with 1000 random token ids over 100 loops, execution time decreased from ~3.14s to ~0.25s, achieving a ~12.5x speedup while preserving bit-exact matching. Signed-off-by: Jules <jules@example.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
💡 **What:** Replaced the highly nested Python loops inside `_simhash_impl` with vectorized NumPy operations and fast bit-packing using `np.packbits` and `int.from_bytes()`. 🎯 **Why:** The existing implementation iterated bitwise 64 times for every token purely in Python, causing enormous performance degradation during scaling or long sequences. 📊 **Measured Improvement:** Utilizing an internal isolated benchmark with 1000 random token ids over 100 loops, execution time decreased from ~3.14s to ~0.25s, achieving a ~12.5x speedup while preserving bit-exact matching. Signed-off-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
SuarezPM
added a commit
that referenced
this pull request
Jun 4, 2026
Applies Jules PR #83 (uv.lock/log churn dropped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Pablo <suarezpm@csnat.unt.edu.ar>
Owner
Author
|
Applied to |
Contributor
Acknowledged. |
SuarezPM
added a commit
that referenced
this pull request
Jun 4, 2026
PR #83 vectorized _simhash_impl in embeddings/__init__.py, but that module's EmbeddingEngine is a dead duplicate (no importer; the 3 real consumers use embeddings.embedding_engine). This ports the numpy/packbits impl to the LIVE embedding_engine._simhash_impl where simhash actually runs. Output verified identical to the old loop; ~15x faster on CPU (16-4096 tokens, Ryzen 5 3600). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Pablo <suarezpm@csnat.unt.edu.ar>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 What: Replaced the highly nested Python loops inside
_simhash_implwith vectorized NumPy operations and fast bit-packing usingnp.packbitsandint.from_bytes().🎯 Why: The existing implementation iterated bitwise 64 times for every token purely in Python, causing enormous performance degradation during scaling or long sequences.
📊 Measured Improvement: Utilizing an internal isolated benchmark with 1000 random token ids over 100 loops, execution time decreased from ~3.14s to ~0.25s, achieving a ~12.5x speedup while preserving bit-exact matching.
PR created automatically by Jules for task 18412311951160534470 started by @SuarezPM