⚡ Remove unused heap logic from VRAMAwareCache#81
Closed
SuarezPM wants to merge 4 commits into
Closed
Conversation
Removes the unused `self._heap` and associated `_reheap` logic from `VRAMAwareCache`. Eviction in this class determines items to remove by iterating or sorting `self._store.values()`, bypassing the heap completely. Re-heaping on every eviction was adding an O(N) overhead for no functional benefit. Measured performance improvement in benchmark scenario (1,000,000 entries triggering a single eviction) drops `_apply_eviction_policy` time from 0.81s down to 0.11s. Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
Contributor
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
Removes the unused `self._heap` and associated `_reheap` logic from `VRAMAwareCache`. Eviction in this class determines items to remove by iterating or sorting `self._store.values()`, bypassing the heap completely. Re-heaping on every eviction was adding an O(N) overhead for no functional benefit. Measured performance improvement in benchmark scenario (1,000,000 entries triggering a single eviction) drops `_apply_eviction_policy` time from 0.81s down to 0.11s. Signed-off-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
Removes the unused `self._heap` and associated `_reheap` logic from `VRAMAwareCache`. Eviction in this class determines items to remove by iterating or sorting `self._store.values()`, bypassing the heap completely. Re-heaping on every eviction was adding an O(N) overhead for no functional benefit. Measured performance improvement in benchmark scenario (1,000,000 entries triggering a single eviction) drops `_apply_eviction_policy` time from 0.81s down to 0.11s. Signed-off-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
Removes the unused `self._heap` and associated `_reheap` logic from `VRAMAwareCache`. Eviction in this class determines items to remove by iterating or sorting `self._store.values()`, bypassing the heap completely. Re-heaping on every eviction was adding an O(N) overhead for no functional benefit. Measured performance improvement in benchmark scenario (1,000,000 entries triggering a single eviction) drops `_apply_eviction_policy` time from 0.81s down to 0.11s. Signed-off-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: SuarezPM <110942776+SuarezPM@users.noreply.github.com>
SuarezPM
added a commit
that referenced
this pull request
Jun 4, 2026
Applies Jules PR #81 (uv.lock/log churn dropped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Pablo <suarezpm@csnat.unt.edu.ar>
Owner
Author
|
Applied to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 What: Removed$O(N)$ re-heaping penalty during evictions) that was completely unused. Eviction logic relies entirely on analyzing
self._heap, the_reheapmethod, andheapq.heappushcalls fromVRAMAwareCache.🎯 Why: The cache never actually queried or popped from the heap, meaning it was maintaining a data structure (with an
self._store.📊 Measured Improvement: In a local benchmark isolating
_apply_eviction_policywith 1,000,000 cache items, the execution time for an eviction cycle dropped from ~0.81s to ~0.11s.PR created automatically by Jules for task 12314248908657311996 started by @SuarezPM