Skip to content

research: make the Qwen3.8 RTX 3090 benchmark reproducible on Windows - #3

Draft
MaxKerkula wants to merge 3 commits into
research/qwen38-3090-ple-mmap-feaeaa3from
research/qwen38-3090-benchmark-ready
Draft

research: make the Qwen3.8 RTX 3090 benchmark reproducible on Windows#3
MaxKerkula wants to merge 3 commits into
research/qwen38-3090-ple-mmap-feaeaa3from
research/qwen38-3090-benchmark-ready

Conversation

@MaxKerkula

@MaxKerkula MaxKerkula commented Sep 1, 2026

Copy link
Copy Markdown
Owner

This prepares the exact next RTX 3090 comparison without touching the production vision branch.

Changes:

  • expose physical sequence capacity and KV reserve instead of hard-coding 8192 + decode;
  • expose Qwen3.8 PLE, expert-load, attention, and NVFP4 backend selections;
  • force the spawned server to import the current checkout instead of an editable install from another branch;
  • allow a writable benchmark temp directory for restricted Windows sessions;
  • stop only the spawned Windows process tree;
  • record the full tested geometry in JSON;
  • add focused standard-library tests for command construction, source/temp isolation, defaults, and child-process launch.

Intended fixed run:

python benchmarks/bench_decode_moe.py `
  --model C:\Users\MaxKe\Models\Qwen3.8-Flash-Next-NVFP4-FTW `
  --backend offload --nvfp4-backend triton --ple-backend mmap `
  --expert-load serial --attention-backend qsa_sparse `
  --cache 2048 --max-seq-len 131072 --kv-reserve-tokens 131072 `
  --decode 256 --greedy --temp-dir .bench-temp `
  --json qwen38-128k-current.jsonl

Current verification:

  • GitHub Actions run 33476449285 completed successfully on windows-latest.
  • Python byte-compilation passed.
  • All 5 focused benchmark-harness tests passed.
  • The forced qsa_sparse backend is registered in this branch.

Next fixed gate:

  1. Run the command above on the local RTX 3090.
  2. Require coherent greedy output and clean shutdown of only the spawned server tree.
  3. Compare sustained decode with the existing exact 128K Marlin result of 7.6571328304 tok/s.
  4. Do not add the KV ladder or MTP before this control, because either would make the result impossible to attribute.

The branch already contains mmap PLE and upstream's fast Triton top-10 router. Current external evidence makes correct Qwen3.8 MTP recurrent-state rollback the strongest later route toward 50 accepted tok/s, but this PR makes no throughput or 50 tok/s claim until the local control completes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant