docs: publish M5 Max speed and compression research - #2
Closed
PhilipJohnBasile wants to merge 1 commit into
Closed
Conversation
PhilipJohnBasile
marked this pull request as ready for review
August 11, 2026 12:08
Owner
Author
|
Public CI follow-up: the fork did not auto-create pull_request runs, so I manually dispatched the existing release workflow against the exact PR head |
Owner
Author
|
@youssofal — GitHub would not retain a formal reviewer request on this fork, so I am requesting review here. The branch is evidence-only and changes no runtime defaults. The fastest review path is the main research note plus the content-addressed bundle manifest; the public macOS build/smoke run is linked above. |
This was referenced Aug 11, 2026
Owner
Author
|
Archiving this stacked 2.4-era research PR with its retired parent integration branch. It is not part of the current 2.9.1 mainline or upstream PR youssofal#336. All source, receipts, and commits remain preserved; any still-useful research can be republished later as a focused current-main documentation PR. |
PhilipJohnBasile
pushed a commit
that referenced
this pull request
Sep 1, 2026
…wnership to the engine (Discover order + F20) Forge->Discover was hiding the 3.8 family and community MTPLX models for two reasons — neither was an architecture allowlist (none exists in the pipeline; hypothesis refuted with receipts): 1. Stale name filter: '-MTPLX-' case-sensitive infix match, a fossil of the retired <base>-MTPLX-<role> branding. Forge itself now brands <base>-MTPLX (suffix), so the filter dropped the app's own output plus every lowercase/prefix community variant — 36 of the 109 live MTPLX repos on 2026-08-16, including the #2-by-downloads community model and the project's own Qwen3.5-4B-Optimized-MTPLX. Now: case-insensitive 'mtplx' anywhere in the repo NAME segment (the founder's contract). 2. Slice-before-filter windowing: HF rows were cut to limit+offset before filtering with no pagination, so nothing below rank ~30 by downloads could ever surface — exactly where fresh models (the whole 3.8 family) sit. Now: iterate the downloads-sorted stream and collect until limit cards SURVIVE the filter (scan bounded at 1000 rows); app wall 30->100. Live e2e (the exact command the app spawns): all six official 3.8 artifacts, the community Heretic build (rank 14), and the nom666/samuelfaj packs all return. F20: the app emits --profile only for an engine-launchable value. 'auto' (and unknown strings, which previously coerced to sustained) emit NO flag: the engine owns default-profile resolution, resolves identity from the artifact (youssofal#268), promotes flagships to turbo, and reports the result on /health. The old app-side resolution pinned sustained for legacy hybrids and renamed dirs the path-substring table missed — the hostile-25.3-tok/s mechanism. Safety proof: every family the app pinned turbo for is in the engine's _TURBO_DEFAULT_PUBLIC_MODEL_IDS serve-path set. Rider: ChipTier maps the engine's 'intel' generation string (previously fell to .unknown -> modern-Apple treatment on Intel Macs). 575 XCTest 0 failures; swift build clean; 6-test discover-filter pytest with live-captured HF fixtures; CLI parity dry-run pins app-flagless + engine-resolved halves. Residual (gate follow-ups): cli.py forge-discover --limit default 20; engine per-call clamp 100 leaves ~9 sub-rank-100 tail repos off-wall.
PhilipJohnBasile
pushed a commit
that referenced
this pull request
Sep 1, 2026
…imulator (H2 gate pricing without the tree) Extends the Δ-telemetry (3682543) from unconditioned correction-rank counting into the number HYPER-PLAN Phase-3 H2 actually gates on: at margin threshold T, how often would a B2 fork fire, how often would its second branch catch the rejection, and how many extra committed tokens per verify round would it have been worth. Mechanism (mtplx/forkev_telemetry.py, fully documented there): - margin = q1 - q2 of the draft sampler's shaped SparseDistribution — the exact q acceptance already uses, host NumPy, zero extra evals. - hit = residual correction == draft #2 candidate (same proxy as the n=21 Δ receipts, so conditioned and unconditioned numbers stay comparable). - saved tokens resolved against the NEXT round's accepted run a', priced on a SAME-DEPTH tree (branch2 owns h = K - r continuation slots): saved_lo = 0 if h==0 else 1 + min(a', h-1) [primary, conservative] saved_hi = 0 if h==0 else 1 + min(a', h) [bracket] The structural +1 is the row after the hit token — inside the verify forward either way; the caps burn the leaf bonus and one run slot to offset a''s shallow-depth optimism. Hit token itself never counted. - policy sweep T in {0.05,0.1,0.2,0.3,0.5} (MTPLX_FORKEV_THRESHOLDS overrides), both single-fork-at-first-trigger and fork-at-rejection variants, plus margin-decile x depth sufficient statistics. Wiring: one recorder per generate_mtpk call (None when the env gate is off — a single is-not-None check per round on the hot path), one observe_round at the accept-outcome boundary, finalize + stderr one-liner at stats build. Emission: GenerationStats.forkev -> request-log rows (harvest_cell-readable) and the flight recorder end event; both byte-identical when the gate is off. Grammar-clamped rejections and non-sparse draft rows are excluded and counted, never silently dropped.
PhilipJohnBasile
pushed a commit
that referenced
this pull request
Sep 1, 2026
…ed model) Reuses the context-copy scripted-model harness with a draft head that deliberately disagrees with the target head (target 0.7/0.3 on tokens (2,3), MTP head flipped): probability-ratio acceptance rejects for real, residual corrections land on the draft's #2 candidate, and the whole recorder runs through the actual accept-loop hook — rounds/rejections/hits nonzero, errors 0, margins sparse-materialized (margin_unavailable 0), decile-4 binning and threshold gating exact. The decisive assertion: same seed with MTPLX_FORKEV_TELEMETRY on vs off produces byte-identical token streams — trajectory neutrality proven in code, not by argument — and stats.forkev is {} when off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
test_no_mlx_imports.py,test_public_cli.py,test_runtime_kpis.py)git diff --checkpassedscripts/fresh_venv_smoke.sh1578f6f8cb860be2e6dcb27ea08f1253b069555f0e83da3c71a451a0bcf98d33Evidence limits
Review entry points
docs/research/mlx-m5max-speed-without-quality-loss-20260810.mdbenchmarks/results/mlx-m5-research-20260810/summary.jsonbenchmarks/results/mlx-m5-research-20260810/bundle_manifest.jsondocs/research/mlx-lm-1709-exactness-audit-20260810.md