Skip to content

feat(qwen35): gate speculation by measured acceptance - #624

Closed
Graffioh wants to merge 8 commits into
Luce-Org:mainfrom
Graffioh:codex/acceptance-gated-speculation
Closed

feat(qwen35): gate speculation by measured acceptance#624
Graffioh wants to merge 8 commits into
Luce-Org:mainfrom
Graffioh:codex/acceptance-gated-speculation

Conversation

@Graffioh

@Graffioh Graffioh commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Stack on feat(qwen36): complete concurrent serving implementation with speculation #605 and carry forward its packed mixed DDTree/AR verification, fused replay, and direct-commit execution substrate.
  • Use one measurement-derived score per request: optimistic during probation or stale re-probing, otherwise accepted-token EMA.
  • Sort requests by score and admit the speculative prefix only while each additional lane improves predicted goodput, the route remains at least 5% above AR, and AR-peer latency stays within 110%.
  • Keep only three persistent request measurements: EMA yield, observed rounds, and progress at the last successful speculative round. There are no content labels, feature vectors, trained classifiers, modes, or demotion counters.
  • Preserve per-request adaptive, ar, and speculation controls plus the fixed-policy and forced-oracle test overrides.

Verification

  • HIP Release configure for gfx1151
  • dflash_server and affected engine/unit targets build successfully
  • focused gate, mixed-path, scheduler-contract, snapshot, and decode-mode suite: 9/9 passed
  • complete server unit suite: 366/366 passed
  • whitespace and superseded-policy audits are clean

Notes

This is stacked on the current #605 head (e3cee32). The M3 model-backed throughput/TTFT benchmark matrix has not been run yet and remains follow-up validation for this draft.

Add the packed mixed DDTree/AR execution substrate, a model-neutral acceptance-and-cost gate, and per-request decode modes. Replace cohort thresholds and scout policy with bounded in-band probation, measured break-even routing, hysteresis, and re-probing.
Derive every request score from measured EMA freshness, remove the mode and demotion state machine, and admit only lanes that improve prefix goodput. Carry successful decode progress in observations and cover probation, staleness, EMA hysteresis, noisy costs, and freeloader rejection.
@Graffioh Graffioh closed this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant