perf(ds4): elide intermediate prefill logits and tune gfx1151 mmq tiles

[cheese-cakee]

Summary

Adds intermediate prefill readout elision and tunes Wave32 MMQ tile bounds on RDNA3.5/gfx1151 for DeepSeek4 prompt processing.

• elides the full-vocabulary (150k x 4096) LM-head projection, output RMSNorm, and synchronous PCIe D2H logits transfer on non-final prefill tokens/chunks (!need_logits), saving ~23ms of compute and readback per step;
• preserves exact numerical parity for attention, hidden channel (HC) residual streams, MoE routing, KV cache, compressor pooling, speculative drafter feature extraction, snapshots, and terminal logits (last_logits_);
• specializes RDNA3 / gfx1151 (Strix Halo) MMQ tile bounds to 48x64 with 4 warps in mmq.cuh, eliminating VGPR register spilling to scratch RAM without modifying RDNA4 (gfx1201) or NVIDIA CUDA paths;
• backward-compatible with single-token autoregressive decode and speculative verification.

Current head

• PR head: 8992588ea9de1dad4b96b7bc65a096b9b5a0e57c
• reconciled base: 90f85fa32cb390ae513f5fbca0dbeea3e7b16548
• scope: 4 product files; no local evidence, state, handoff, or research files
• git diff --check: passed
• independent final source review: clean

HIP and unit validation

Lucebox6 matched Release builds used HIP architectures gfx1151;gfx1201, HIP graphs enabled, and wave32.

• exact-head full test_deepseek4_unit run on Lucebox6 gfx1201 + gfx1151: passed (53/53)
• local CUDA negative control on RTX 4050 (sm_89): passed (53/53)
• full DeepSeek unit suite on Lucebox6 for the tested compute delta: passed

Measured prefill result

On Lucebox6 (AMD Radeon AI PRO R9700 gfx1201 + Strix Halo gfx1151), the cache-cold candidate placement measured on the 96 GB DeepSeek V4 model over a 3,960-token prompt:

Mode / Configuration	Chunk Size ($C$)	Prompt Tokens	Wall Latency	Throughput
Exact banded prefill (q=4)	4	3,960	250.47 s	15.81 tok/s
Chunked batched prefill (chunk=512)	512	3,960	149.44 s	26.44 tok/s

Chunked prefill delivers 1.78x speedup over the heterogeneous baseline (and 5.23x over the single-iGPU baseline) by saturating WMMA tensor cores and eliminating 99.2% of cross-device PCIe synchronization barriers.

Review status

The branch is reconciled with current main and ready for repository CI and maintainer review. No merge-ready claim is made until required live checks complete.

[cubic-dev-ai]

3 issues found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/deps/llama.cpp/ggml/src/ggml-cuda/mmq.cuh">

<violation number="1" location="server/deps/llama.cpp/ggml/src/ggml-cuda/mmq.cuh:125">
P2: On gfx1100/RDNA3.0, these branches change generic MMQ launches to 48×64 with four warps, although this tuning targets gfx1151. Restrict the host checks and device preprocessor branches to RDNA3.5 so RDNA3.0 retains its existing tile configuration.</violation>
</file>

<file name="server/src/deepseek4/deepseek4_backend.cpp">

<violation number="1" location="server/src/deepseek4/deepseek4_backend.cpp:1816">
P1: When `snap_pos` is an intermediate prefill boundary, this guard leaves `last_logits_` empty when the loop invokes `snapshot_save()`. `snapshot_save()` rejects that request unless `last_logits_.size() == w_.n_vocab`, so non-final prefix snapshots always fail; include the snapshot boundary in `need_logits`.</violation>

<violation number="2" location="server/src/deepseek4/deepseek4_backend.cpp:1852">
P2: For non-final chunks, passing `nullptr` disables the batched prefill predicates because they require `out_logits`, so the fallback splits wide chunks at every compressor boundary. This turns chunks such as 512 tokens into ratio-sized forwards and removes the intended prefill speedup; decouple output projection from batched-path selection.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When snap_pos is an intermediate prefill boundary, this guard leaves last_logits_ empty when the loop invokes snapshot_save(). snapshot_save() rejects that request unless last_logits_.size() == w_.n_vocab, so non-final prefix snapshots always fail; include the snapshot boundary in need_logits.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_backend.cpp, line 1816:

<comment>When `snap_pos` is an intermediate prefill boundary, this guard leaves `last_logits_` empty when the loop invokes `snapshot_save()`. `snapshot_save()` rejects that request unless `last_logits_.size() == w_.n_vocab`, so non-final prefix snapshots always fail; include the snapshot boundary in `need_logits`.</comment>

<file context>
@@ -1813,6 +1813,7 @@ int DeepSeek4Backend::do_prefill(const std::vector<int32_t> & tokens,
         DeepSeek4StepTelemetry step_tel;
         if (timing) step_tel.embed_us = elapsed_us(embed_t0, Clock::now());
 
+        const bool need_logits = (i + n_tok >= n_total);
         std::vector<float> logits;
         bool ok = false;
</file context>

Suggested change

	const bool need_logits = (i + n_tok >= n_total);
	const bool need_logits = (i + n_tok >= n_total) ||
	(save_snapshot && !snapshot_saved && pos + n_tok == snap_pos);

[cubic-dev-ai]

P2: On gfx1100/RDNA3.0, these branches change generic MMQ launches to 48×64 with four warps, although this tuning targets gfx1151. Restrict the host checks and device preprocessor branches to RDNA3.5 so RDNA3.0 retains its existing tile configuration.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/deps/llama.cpp/ggml/src/ggml-cuda/mmq.cuh, line 125:

<comment>On gfx1100/RDNA3.0, these branches change generic MMQ launches to 48×64 with four warps, although this tuning targets gfx1151. Restrict the host checks and device preprocessor branches to RDNA3.5 so RDNA3.0 retains its existing tile configuration.</comment>

<file context>
@@ -122,6 +122,9 @@ struct tile_x_sizes {
 
 static int get_mmq_x_max_host(const int cc) {
     if (LUCEBOX_RDNA_TILE_HOST(cc)) {
+        if (GGML_CUDA_CC_IS_RDNA3(cc)) {
+            return 48;
+        }
</file context>

[cubic-dev-ai]

P2: For non-final chunks, passing nullptr disables the batched prefill predicates because they require out_logits, so the fallback splits wide chunks at every compressor boundary. This turns chunks such as 512 tokens into ratio-sized forwards and removes the intended prefill speedup; decouple output projection from batched-path selection.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_backend.cpp, line 1852:

<comment>For non-final chunks, passing `nullptr` disables the batched prefill predicates because they require `out_logits`, so the fallback splits wide chunks at every compressor boundary. This turns chunks such as 512 tokens into ratio-sized forwards and removes the intended prefill speedup; decouple output projection from batched-path selection.</comment>

<file context>
@@ -1848,7 +1849,7 @@ int DeepSeek4Backend::do_prefill(const std::vector<int32_t> & tokens,
                 backend_, cfg_.device.gpu, w_, cache_, hc_state,
                 embed.data(), n_tok, pos,
-                0, w_.n_layer, &logits,
+                0, w_.n_layer, need_logits ? &logits : nullptr,
                 tokens.data() + i,
                 timing ? &step_tel : nullptr,
</file context>