Feat performance#20
Merged
Merged
Conversation
added 30 commits
June 13, 2026 07:28
- tests/perf/string_corpus.py: 10 family generators with acceptance checks - tests/perf/test_string_corpus.py: 58 tests for all families at sizes 32, 128, 1024 - All generators are deterministic and satisfy plan acceptance criteria - FAMILY_REGISTRY exports all 10 families - DEFAULT_SIZES and EXTENDED_SIZES constants defined
- tests/perf/harness.py: measure_call, assert_within_budget, scaling_rows, classify_rows - tests/perf/test_harness.py: 24 self-tests for timing, scaling, and allocation - Default budget is 100ms (PARSING_BUDGET_MS override) - Default scaling ratio threshold is 3.0 (PARSING_SCALING_RATIO_MAX override) - Linear functions classify as 'pass' - Quadratic functions classify as 'superlinear' - Large allocations classify as 'allocation_exceeded' - Uses tracemalloc for peak memory tracking
- tests/perf/registry.py: 16 entries mapping review hotspots to single-string wrappers - tests/perf/test_parsing_smoke.py: 24 smoke tests calling each entry with plain_ascii(1024) - Each entry has name, component, call, notes, and is_decode_path fields - Registry covers: parse_json_type, datetime parsing, number affix, base64, color detection, text inference, compute_editable, format_with_type, decode_bytes - Smoke tests pass under make test without the perf marker
- tests/perf/test_parsing_scaling.py: @pytest.mark.perf tests parametrized by registry entry, family, and size sequence - With PYTEST_PERF_STRICT unset, records rows without failing - With PYTEST_PERF_STRICT=1, fails on budget_exceeded, superlinear, allocation_exceeded, or error outcomes - Produces one row for every registry/family/size combination - Collects rows for report generation in Commit 0.8
- tests/perf/test_regex_backtracking.py: @pytest.mark.perf tests for DATETIME_RE, _CURRENCY_RE, _UNITS_RE, _B64_RE, looks_like_color_rgb, and looks_like_color_rgba - Uses near_datetime, near_affix, near_color, and pathological_repetition families - Each regex measured with fullmatch or existing public wrapper - Near-miss verification tests run in default suite (not perf-marked) - Strict mode fails any regex row with scaling ratio > 3.0 or budget exceeded
- tests/perf/test_decode_amplification.py: @pytest.mark.perf tests for _looks_like_base64, parse_json_type, compute_editable, decode_bytes, and format_with_type with BYTES/ZLIB/GZIP branches - Valid small fixtures exercise successful decode paths - base64_like and plain_ascii families test scaling behavior - Oversized text tests verify no crash on large inputs - Report rows distinguish time budget from allocation failures
- tests/perf/test_container_paths.py: @pytest.mark.perf tests for format_with_type with STRING, MULTILINE, and UNICODE types - Smoke tests for formatting run in default suite - Filter proxy tests deferred due to QCoreApplication conflicts - Report contains rows for formatting at each default size - Test fixture creates no tabs and no application-level state
- tests/perf/report.py: report writer for parsing vulnerability measurement - reports/parsing-vulnerability-2026-06-13.md: 640 rows across 16 registry entries and 10 adversarial families at sizes 1024, 4096, 16384, 65536 - plans/01-string-parsing-len-limits.md: updated threshold table with report citations - plans/00-parsing-vulnerability-tests.md: marked Commit 0.8 as completed - pytest.ini: registered 'perf' marker for opt-in performance tests - tests/perf/test_harness.py: fixed flaky linear classification test Report summary: - 487 pass, 121 superlinear, 32 errors (decode_bytes on non-base64 input) - 0 budget exceeded, 0 allocation exceeded - All functions pass at 65536 except decode_bytes (expected errors on invalid input) - Strict performance failures are opt-in and not part of make test
- tests/perf/string_corpus.py: Added 3 new families: - trace_repetition: TRACE_EXAMPLE repeated for realistic error trace content - source_code_repetition: SOURCE_CODE_EXAMPLE repeated for realistic code content - escape_heavy: JSON_TO_BE_ESCAPED with double/triple escaping for backslash stress - tests/perf/test_string_corpus.py: Added 16 new tests for the new families - reports/parsing-vulnerability-2026-06-13.md: Regenerated with 832 rows (up from 640) - 647 pass, 141 superlinear, 44 errors (decode_bytes on non-base64) - New families show expected behavior (errors on invalid base64 input) - plans/01-string-parsing-len-limits.md: Updated report citation with new row counts The new families provide more realistic performance testing with: - Multi-line error traces (trace_repetition) - Source code with imports, functions, and control flow (source_code_repetition) - Heavy escape sequences with backslashes (escape_heavy)
…rs, explicit coercion bypass seam, datetime and affix gating - settings.py: Add INFERENCE_MAX_DATETIME_CHARS (40), INFERENCE_MAX_AFFIX_CHARS (100), INFERENCE_MAX_COLOR_CHARS (10), FORMAT_PREVIEW_DECODE_LIMIT_BYTES (100) - tree/inference_limits.py: New module with datetime_inference_allowed, affix_inference_allowed, color_inference_allowed, base64_syntax_valid, format_preview_decode_allowed - core/datetime_parsing/regex.py: Add allow_expensive param to parse_datetime_text, gate regex by length - units/number_affix.py: Add allow_expensive param to parse_number_affix, gate regex by length - tree/item_coercion.py: Add allow_expensive param to coerce_value_for_type, propagate through temporal and affix helpers - tree/item.py: Pass allow_expensive=self.explicit_type from _coerce_value_for_type - Tests: test_inference_constants, test_inference_limits, test_explicit_type_bypass, test_datetime_inference_limits
- tree/types.py: Add allow_expensive param to looks_like_color_rgb/rgba, gate by INFERENCE_MAX_COLOR_CHARS (10) - Explicit color coercion via normalize_color_string uses parse_color which bypasses the gate naturally
- tree/types.py: Refactor _looks_like_base64 to use base64_syntax_valid as cheap pre-check - base64_syntax_valid checks len mod 4 + alphabet regex before expensive b64decode - Minimum 20 chars retained to avoid false positives on short strings - No length cap on valid base64: if syntax passes, decode proceeds
- delegates/formatting/value_formatting.py: Cap decode work at FORMAT_PREVIEW_DECODE_LIMIT_BYTES (100 bytes) - For small strings: decode fully for accurate size display - For large strings: decode only enough base64 chars for 16-byte preview - Estimate total size from base64 length (accounting for padding) - Show truncation marker for oversized previews
- Create app/loading/coordinator.py with LoadCoordinator class - Route _open_path() and _reload_tab_from_path() through coordinator - Add tests for coordinator open and reload methods - Existing open/reload behavior unchanged
added 29 commits
June 13, 2026 16:27
- Create app/loading/builder.py with ChunkedTreeBuilder class - Builder wraps synchronous build with chunked progress reporting - Event loop remains responsive during large builds - Add tests for fixture comparison, event loop, and no partial model
- Add stage_changed signal to LoadCoordinator - Add set_reporter() and _emit_stage() methods - Add run_schema_discovery_and_validation() with stage reporting - Add tests for stage emission and reporter integration
- Integrate LoadingProgressDialog with LoadCoordinator - Add _start_progress, _finish_progress, _error_progress methods - Open and reload operations now emit all progress stages - Progress widget hidden for fast operations, shown for slow ones - Add end-to-end tests for stage emission and widget visibility
…t 2.8) - Use STAGE_APPLYING_RELOAD for reload instead of STAGE_BINDING_UI - Add tests for reload stage emission and data integrity - Verify undo stack cleared on change, preserved on no change - Verify old data preserved on reload error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.