Release 3.2.0: issue #5 performance work - #6
Merged
Conversation
Decompression dominated project opening (88-96% per the issue's profile). Two changes, both output-preserving: decompress now produces output with slice operations wherever the spec allows: non-overlapping copy tokens (offset >= length) move as one slice, runs of literal tokens within a flag byte extend once, and the copy-token masks recompute only when the chunk-local output size crosses a power of two instead of once per token. Overlapping copies keep the spec's byte-at-a-time semantics, which are load-bearing there. Measured 12.4 -> 21.8 MB/s over the 31 module and dir streams in the live fixtures. A new oracle test suite pins the optimized decoder against the original per-byte implementation: live-fixture streams, synthetic round trips (overlap-heavy included), and byte-equal error messages and offsets on malformed input. A new max_bytes parameter returns a chunk-aligned prefix; chunk locality (enforced by this decoder) makes the prefix byte-identical to the same range of a full decompression. parse_vba_project now decompresses only the first chunk of each module stream. That prefix carries the Attribute header, and for single-chunk modules it already is the complete source, which stays eager at zero extra cost. Multi-chunk modules defer the remaining chunks to the first VBAModule.source access via a stored loader; stream lookup and MODULEOFFSET bounds checks remain eager, and a header that runs to the chunk boundary falls back to eager full decompression rather than risk a truncated attribute_header. VBAModule becomes a regular class with an unchanged constructor signature plus a source_loaded property; dataclass field equality and repr are gone (equality is identity). Opening the large-module fixture for module_names() drops 1.47 ms -> 0.79 ms, and saving an unrelated edit no longer decompresses untouched modules (tested). Verified byte-identical end to end: the 25-case save matrix across all live fixtures hashes identically against a v3.1.0 worktree baseline, and the live Excel gate passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements both halves of issue #5, byte-exactness preserved throughout.
Verification: 390 tests + pyright strict + ruff clean; 25-case save matrix byte-identical against a v3.1.0 worktree baseline; live Excel gate green.
Closes #5.