feat(qwen36): complete concurrent serving implementation with speculation

harness/benchmarks/concurrency/FEATURE_MATRIX.md

@@ -0,0 +1,131 @@
+# Qwen3.6 concurrent feature matrix
+
+The bounded Strix Halo measurements collected for the draft implementation are
+recorded in [`STRIX_HALO_RESULTS.md`](STRIX_HALO_RESULTS.md).
+
+`run_qwen36_feature_matrix.sh` extends the PR #596 protocol with feature
+ablations for the complete Strix Halo configuration:
+
+- `ar`: concurrent paged autoregressive control.
+- `ddtree`: adds the decode draft, DDTree, and the recorded budget.
+- `pflash`: adds auto prefill compression, its drafter, and persistent draft
+  residency.
+- `kvflash`: adds bounded KV residency in auto mode and explicitly supplies
+  the hashed prefill drafter for relevance-scored page selection; prefill
+  compression remains off in this ablation.
+- `full`: enables DDTree, PFlash, and KVFlash together with both devices on
+  `hip:0`.
+- `llama`: optional external comparison; its binary is required only when this
+  variant is explicitly requested.
+
+Run the default bounded C4 screening repeat (seven applicable fresh-server cases):
+
+```bash
+MODEL=/opt/models/Qwen3.6-27B-Q4_K_M.gguf \
+DRAFT_MODEL=/opt/models/draft/dflash-draft-3.6-q8_0.gguf \
+PREFILL_DRAFTER=/opt/models/Qwen3-0.6B-BF16.gguf \
+REPEATS=1 \
+harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+```
+
+For the canonical AMD Strix Halo recipe, use the published Q8_0 3.6 drafter and make the tuning explicit:
+
+```bash
+MODEL=/opt/models/Qwen3.6-27B-Q4_K_M.gguf \
+DRAFT_MODEL=/opt/models/draft/dflash-draft-3.6-q8_0.gguf \
+PREFILL_DRAFTER=/opt/models/Qwen3-0.6B-BF16.gguf \
+DRAFT_SWA=2048 PREFILL_UBATCH=512 DDTREE_ADAPTIVE=0 \
+VARIANTS=ddtree,pflash,kvflash,full CLIENTS=1,4,8,16 REPEATS=5 \
+harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+```
+
+`DDTREE_ADAPTIVE=0` matches the blog's continuous DDTree probe policy; leave it at the default `1` when measuring the concurrent engine's adaptive fallback policy. The concurrent path now records a startup `[parallel-ddtree]` marker and per-request `ddtree_steps`; these are the proof that DDTree actually ran.
+
+On a 128 GiB Strix Halo host, budget roughly 45–90 minutes for this smoke run;
+the long-context AR controls dominate and actual time depends on the build.
+Every row remains independently selectable through `VARIANTS`. For example:
+
+```bash
+WORKLOADS=short CLIENTS=4 VARIANTS=ar,ddtree MAX_TOKENS=256 REPEATS=1 \
+harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+```
+
+Use five fresh-process, paired repeats for published measurements:
+
+```bash
+WORKLOADS=short,compression CLIENTS=1,4,8,16 \
+VARIANTS=ar,ddtree,pflash,kvflash,full MAX_TOKENS=256 REPEATS=5 \
+harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+```
+
+That full matrix can take roughly 15–30 hours on Strix Halo; keep the generated
+case directory so interrupted or suspect rows can be diagnosed rather than
+quoted.
+
+## Activation workloads
+
+Auto features cannot be validated with the original 400–4,000 word prompts.
+The extension adds two deterministic, disjoint 29-prompt cohorts:
+
+- `compression`: 34K–40K words, chosen after observing 38,130–44,856
+  tokens with the development Qwen GGUF tokenizer.
+- `kv-pressure`: 12K–18K words, which produced 13,463–20,190 tokens with
+  that tokenizer against the runner's explicitly recorded 8K pool cap.
+
+The observed counts are a fixture sanity check, not a claim derived from word
+count and not publication evidence. Each row records the model hash; the proof
+cross-checks logged raw PFlash input against `usage.prompt_tokens`, requires
+it to meet the recorded auto threshold, and uses server-reported effective
+tokens plus actual page traffic for KVFlash.
+
+The bounded default uses `short,compression` at C4. It runs PFlash and the
+full configuration only on `compression`; KVFlash can also be selected on
+`kv-pressure`. Inapplicable pairs are printed as skips and never appear as
+successful rows. AR controls use the same prompts, so feature deltas remain
+paired.
+
+## Fail-closed feature proof
+
+The server must write one JSON object per completed request with this prefix:
+
+```text
+[concurrency-metrics] {"request_id":"...", ...}
+```
+
+Required fields are `effective_prompt_tokens`, `ddtree_steps`,
+`ddtree_suspensions`, `ddtree_accepted_tokens`, `target_forwards`,
+`kvflash_page_ins`, `kvflash_page_outs`, `kvflash_resident_blocks`,
+`kvflash_reselects`,
+`pflash_applied`, `pflash_input_tokens`, and `pflash_output_tokens`.
+
+The proof tool correlates log objects with measured SSE request IDs and also
+checks the log's effective token count against
+`usage.timings.effective_prompt_tokens`. A requested feature fails the case
+unless:
+
+- DDTree has positive step and target-forward counts. Acceptance may be zero.
+  The required per-request suspension counter must be either zero or one. It
+  records adaptive fallback activation, but does not prove whether AR work ran
+  before or after the suspension; temporal claims require direct ordered-log
+  evidence.
+- PFlash reports `pflash_applied=true`, a smaller output prompt, and (in auto
+  mode) an input token count at or above the recorded activation threshold.
+- KVFlash always records an explicit hashed scorer drafter, reports its
+  startup-observed physical pool and enabled metadata, and has a positive
+  resident-block count. `kvflash`-only and `kv-pressure` rows must also show page-in or
+  page-out traffic. For a `full` row, traffic is required only when a
+  server-reported `effective_prompt_tokens` value exceeds that observed pool
+  token limit; zero traffic is valid when PFlash compression fits in the pool.
+
+After health succeeds, the runner fail-closed parses the backend's
+`[parallel-kvflash] physical resident pool ...` and `[paged-attention] ...`
+startup markers into `runtime_observed`. This distinguishes the actual resident
+pool from both the requested auto cap and `--kv-pool-tokens`, which concurrent
+KVFlash intentionally does not use to expand VRAM.
+
+Each case retains the exact shell-escaped command, controlled launch
+environment, literal client process arguments and client-script hash,
+binary/shared-library/target/draft/PFlash-and-KV-scorer hashes, the ordered
+`literal_screenshot_flags` array, all feature values, raw request report,
+server log, and `feature-proof.json`. The summary refuses to
+include a Lucebox row whose proof is missing or invalid.

harness/benchmarks/concurrency/README.md

@@ -0,0 +1,70 @@
+# Qwen3.6 concurrency benchmark
+
+This protocol measures the serving behavior targeted by packed continuous
+prefill and concurrent decode. It is intentionally small: one streaming client,
+one fresh-process runner, one deterministic prompt generator, and one summary
+script.
+
+Run a quick screening repeat:
+
+```bash
+MODEL=/path/to/Qwen3.6-27B-Q4_K_M.gguf \
+LUCE_SERVER_BIN=server/build-hip/dflash_server \
+LLAMA_SERVER_BIN=/path/to/llama-server \
+harness/benchmarks/concurrency/run_qwen36_concurrency.sh
+```
+
+Run a decode-heavy comparison with the same harness:
+
+```bash
+MODEL=/path/to/Qwen3.6-27B-Q4_K_M.gguf \
+LUCE_SERVER_BIN=server/build-hip/dflash_server \
+LLAMA_SERVER_BIN=/path/to/llama-server \
+WORKLOADS=short MAX_TOKENS=256 VARIANTS=luce-k8,llama REPEATS=3 \
+harness/benchmarks/concurrency/run_qwen36_concurrency.sh
+```
+
+The short ragged prompts keep admission realistic while 256 forced output
+tokens make generation dominate the measured window. Use `REPEATS=5` for
+publication. Every measured case starts a fresh server and first runs a
+discarded warmup at the same concurrency. The variants are:
+
+- `luce-k8`: packed prefill with up to eight concurrent prefills.
+- `luce-k1`: the same binary/configuration with packing width limited to one.
+- `llama`: llama.cpp continuous batching with fixed `-b 2048 -ub 512`.
+
+The 29 generated prompts are disjoint cohorts for C1/C4/C8/C16. C4 and above
+contain four substantial length strata while holding the mean target length
+constant. The default short, medium, and long profiles target mean lengths of
+400, 1,000, and 3,000 words per request. Those generator targets are not token
+counts; reports retain the exact server-observed token counts for the selected
+model and tokenizer. The client refuses to wrap or reuse a prompt.
+
+The headline metric is aggregate output goodput: exact server-reported
+completion tokens divided by level wall time. It includes queueing, prefill,
+and decode and must not be called decode throughput.
+
+`Output-window tok/s` divides exact completion tokens by the interval from the
+earliest observed first output to the final request completion. It removes the
+initial all-prefill interval and is decode-facing, but it can still contain
+staggered prefill while later requests await their first token.
+`Request decode tok/s` is the median per-request estimate
+`(completion_tokens - 1) / (end - first_output)`; it assumes the first
+observed streaming event accounts for one token. Neither metric is pure kernel
+decode throughput.
+
+`Prompt tok/s to first` is the sum of server-reported prompt tokens divided by
+the latest first-token arrival; it is a useful prefill-facing metric but still
+includes admission, queueing, and transport. Report TTFT median/max alongside
+all throughput metrics.
+
+The K8-vs-K1 comparison is the causal packing ablation. The K8-vs-llama
+comparison is the product comparison. Five paired repeats, the exact command
+and hashes recorded in each case, zero failures, and a fixed declared output
+length are required before using results in a post. The standard prefill-facing
+protocol uses 64 output tokens; the decode-heavy protocol above uses 256.
+Variant gains are computed as the median of same-repeat ratios, not as a ratio
+of independently aggregated medians. The summarizer rejects mismatched repeat
+sets. It also marks whether each variant produced the same ordered output
+hashes across at least two repeats; a one-repeat screen reports stability as
+`n/a`, and an unstable result is a correctness warning, not a performance win.

harness/benchmarks/concurrency/STRIX_HALO_RESULTS.md

@@ -0,0 +1,134 @@
+# Qwen3.6 concurrent feature results — Strix Halo
+
+- Date: 2026-08-13
+- Implementation: `568fbac03b498d53d6efc0b2ab5893044543a321`
+- Stack base: PR #595 head `a90ffe45c1d4ad58f5f73c4107571d3cf6c51bfd`
+
+These are bounded engineering measurements for the draft PR, not the
+five-repeat publication matrix described in `FEATURE_MATRIX.md`. The paired
+AR/DDTree screen has three fresh-process repeats. The long-context activation
+rows have one fresh-process repeat per concurrency level because they are much
+more expensive; treat their throughput as screening data.
+
+## System and artifacts
+
+- AMD Ryzen AI MAX+ 395 with Radeon 8060S (`gfx1151`), 128 GiB unified memory.
+- ROCm runtime 7.2.4.
+- Release HIP build for `gfx1151` with
+  `DFLASH27B_HIP_SM80_EQUIV=ON`.
+- Server SHA-256:
+  `c77b4d2c7d1505fcc751600a6603cd65e51514b685bc66c4f7d33cd64a87c8a6`.
+- Target SHA-256:
+  `5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0`.
+- Decode draft SHA-256:
+  `e2500e90165a0f8e7b52c9882c29ed1fa391c60b300ff11b817bf10e31fa092e`.
+- PFlash/KV scorer drafter SHA-256:
+  `f9c9f1d3c1e21755b82d4e165f88dbbbd4355646d632fb5d6cef7c66ed4ee04e`.
+
+Every case started a fresh server, discarded an 8-token same-concurrency
+warmup, then requested exactly 64 output tokens per request with temperature
+zero, seed one, and EOS ignored. Prompts were deterministic, disjoint across
+concurrency levels, and identical between paired variants. The runner rotated
+variant order across repeats.
+
+`Output-window` counts all completion tokens from the earliest first output to
+the last completion. `Goodput` counts completion tokens over the whole level,
+including TTFT. Every reported row passed exact token accounting and the
+request-ID-correlated feature proof.
+
+The retained screening artifacts contain maximum TTFT but not median TTFT.
+Their max-only columns below are an explicit screening exception, not a
+protocol-complete publication result; a publication rerun must report both.
+
+## Paired AR and adaptive DDTree
+
+The DDTree configuration adds the local decode draft, budget 22, and target and
+draft placement on `hip:0`. Values are medians over three fresh-process
+repeats.
+
+| C | Variant | N | Goodput tok/s | Output-window tok/s | vs AR goodput | Accepted/step | Steps/suspensions | Max TTFT s | Output hashes stable |
+| ---: | :--- | ---: | ---: | ---: | ---: | ---: | :--- | ---: | :---: |
+| 1 | AR | 3 | 9.41 | 12.57 | — | — | 0/0 | 1.707 | yes |
+| 1 | DDTree | 3 | 9.00 | 11.85 | -4.4% | 1.00 | 1/1 | 1.715 | yes |
+| 4 | AR | 3 | 20.46 | 36.21 | — | — | 0/0 | 5.508 | no |
+| 4 | DDTree | 3 | 19.43 | 33.61 | n/a | 3.08 | 4/4 | 5.533 | no |
+| 8 | AR | 3 | 27.44 | 65.93 | — | — | 0/0 | 11.008 | no |
+| 8 | DDTree | 3 | 25.63 | 56.32 | n/a | 2.79 | 8/8 | 11.076 | no |
+| 16 | AR | 3 | 31.82 | 58.79 | — | — | 0/0 | 22.487 | no |
+| 16 | DDTree | 3 | 29.36 | 54.63 | n/a | 2.00 | 16/16 | 22.514 | no |
+
+The supplied draft had weak acceptance on this cohort. The adaptive policy
+sampled one real packed-tree step, then suspended the whole cohort because its
+aggregate emitted yield was below six tokens per request. At C4 and above,
+the raw timings are retained only to diagnose this fallback behavior; unstable
+outputs do not support a performance comparison with AR.
+
+At C4 and above, greedy text hashes varied across fresh repeats in both the AR
+control and DDTree. C1 was byte-stable. These measurements therefore establish
+exact token accounting and feature execution, but do not claim bitwise text
+reproducibility for concurrent batches.
+
+## Full screenshot configuration
+
+These rows enable the complete requested product configuration:
+
+```text
+--target-device hip:0
+--draft-device hip:0
+--ddtree
+--ddtree-budget 22
+--draft-residency persistent
+--prefill-compression auto
+--prefill-drafter /opt/models/Qwen3-0.6B-BF16.gguf
+--kvflash auto
+```
+
+The controlled runner sets the auto PFlash threshold to 32K tokens, the keep
+ratio to 0.05, and the KVFlash resident cap to 8,192 tokens. Startup telemetry
+confirmed 512 physical blocks of 16 tokens, 16 configured slots, and a 65,536
+logical-token bound per slot.
+
+| C | N | Goodput tok/s | Output-window tok/s | Request decode tok/s | Raw prompt range | Effective prompt range | Max TTFT s | DDTree steps/susp. | KV page in/out | PFlash requests |
+| ---: | ---: | ---: | ---: | ---: | :--- | :--- | ---: | :--- | :--- | ---: |
+| 1 | 1 | 2.61 | 12.39 | 12.19 | 41,504 | 2,021 | 19.324 | 1/1 | 0/0 | 1 |
+| 4 | 1 | 3.00 | 31.61 | 7.95 | 38,142–44,866 | 1,870–2,235 | 77.495 | 4/4 | 1/18 | 4 |
+| 8 | 1 | 3.13 | 52.60 | 6.56 | 38,141–44,870 | 1,869–2,237 | 153.782 | 8/8 | 245/792 | 8 |
+| 16 | 1 | 3.19 | 18.73 | 2.34 | 38,140–44,872 | 1,867–2,238 | 307.360 | 16/16 | 293/1,880 | 16 |
+
+All four rows proved DDTree, PFlash, and KVFlash active. PFlash retained about
+4.9% of raw prompt tokens. Output-window throughput scaled through C8, then
+dropped at C16 while roughly 32K effective prompt tokens shared the 8K resident
+pool; the concurrent page traffic rose accordingly.
+
+## Feature ablations
+
+| Workload | C | Variant | N | Goodput tok/s | Output-window tok/s | Request decode tok/s | Effective/raw | Max TTFT s | Activation evidence |
+| :--- | ---: | :--- | ---: | ---: | ---: | ---: | ---: | ---: | :--- |
+| compression | 4 | PFlash | 1 | 3.15 | 35.54 | 8.89 | 0.049 | 74.261 | 4/4 prompts compressed, 166,016 -> 8,179 tokens |
+| kv-pressure | 4 | KVFlash | 1 | 1.24 | 8.56 | 5.32 | 1.000 | 197.859 | 129 resident blocks max, 0 page-ins / 3,714 page-outs |
+
+The PFlash-only row uses the same C4 prompts as the full row; adding DDTree and
+KVFlash reduced output-window throughput from 35.54 to 31.61 tok/s in this
+single screening repeat. The KVFlash-only row deliberately disables PFlash and
+uses 13,474–20,203-token histories against the 8K pool. It is an activation and
+pressure test, not a recommended latency configuration.
+
+## Reproduction
+
+The exact per-case command, controlled environment, startup-observed pool,
+binary/shared-library/model hashes, raw request report, server log, and
+`feature-proof.json` are retained by the runner. The principal invocations were:
+
+```bash
+WORKLOADS=short CLIENTS=1,4,8,16 VARIANTS=ar,ddtree MAX_TOKENS=64 REPEATS=3 SLOTS=16 harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+
+WORKLOADS=compression CLIENTS=1,4,8,16 VARIANTS=full MAX_TOKENS=64 REPEATS=1 SLOTS=16 harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+
+WORKLOADS=compression CLIENTS=4 VARIANTS=pflash MAX_TOKENS=64 REPEATS=1 SLOTS=16 harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+
+WORKLOADS=kv-pressure CLIENTS=4 VARIANTS=kvflash MAX_TOKENS=64 REPEATS=1 SLOTS=16 harness/benchmarks/concurrency/run_qwen36_feature_matrix.sh
+```
+
+Set `MODEL`, `DRAFT_MODEL`, `PREFILL_DRAFTER`, and `LUCE_SERVER_BIN` as shown
+in `FEATURE_MATRIX.md`. For publication-quality claims, rerun the documented
+five-repeat 256-token matrix.