fix(qwen3-14b): guard negative slot_mapping KV writes

[Leaf-Salix]

Summary

The serving CI for Qwen3-14B can fail during model initialization/warmup with AICore 507018.

After isolating the failure, the warmup path in pypto-serving intentionally fills Qwen prefill/decode slot_mapping with -1:

• prefill warmup uses scratch KV pages and slot_mapping=-1
• decode warmup also fills the decode slot mapping buffer with -1
• the intended meaning is "do not write this token into the real KV cache"

However, the Qwen3-14B kernels were still deriving physical KV cache rows from that negative slot and could issue invalid K/V cache writes. DeepSeek already treats slot_mapping=-1 as a no-write sentinel and skips cache writes when slot < 0, so this PR makes Qwen follow the same contract.

What Changed

• Guard Qwen3-14B prefill K/V cache writes with slot >= 0.
• Guard the generated Qwen3-14B decode RoPE/KV CCE prologue so negative slots skip K/V cache stores.
• Preserve Q output writes and event wait/flag flow; only KV cache writes are controlled by the negative-slot sentinel.
• Apply the same slot < 0 no-write convention to adjacent Qwen A8W8/TQ/SSN draft paths for consistency with the main kernels and DeepSeek.

Tests

All tests used the same clean pypto-serving checkout and only switched the pypto-lib checkout through PYPTO_ROOT / PYTHONPATH.

• Isolated prefill contract, slot_mapping=-1, scratch 1 page:
  • upstream 68889bc: 0/5 PASS, 5/5 AICore 507018
  • fix branch 65ae4d5: 5/5 PASS
• Positive-slot control, slot_mapping=arange, scratch 32 pages:
  • upstream 68889bc: 3/3 PASS
  • fix branch 65ae4d5: 3/3 PASS
• Clean serving accuracy, tests/test_qwen3_accuracy.py:
  • upstream 68889bc: 3/5 PASS, 2/5 AICore 507018
  • fix branch 65ae4d5: 5/5 PASS
• Clean full serving path, tests/test_qwen3_serving.py:
  • upstream 68889bc: 1/5 PASS, 4/5 AICore 507018
  • fix branch 65ae4d5: 5/5 PASS

Full-serving run details for fix branch 65ae4d5:

• iter1: 4 passed, 5 warnings in 119.13s
• iter2: 4 passed, 5 warnings in 118.92s
• iter3: 4 passed, 5 warnings in 121.34s
• iter4: 4 passed, 5 warnings in 120.75s
• iter5: 4 passed, 5 warnings in 118.40s

Notes

• The isolated slot_mapping=-1 contract is the strongest attribution evidence: upstream fails deterministically, the fix passes deterministically, and valid positive slots pass on both branches.
• This is a lib-side contract fix rather than a serving workaround. Serving is already expressing "scratch/no-write" with slot_mapping=-1; Qwen kernels should honor that convention just like DeepSeek.
• The CCE decode change deliberately keeps synchronization behavior outside the sentinel contract. Negative slots skip only K/V cache stores.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e6c483e2-a404-4c51-a0bf-f7ed8f33a30e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Qwen3 14B prefill and decode paths now treat negative slot mappings as invalid and skip KV-cache data and scale writes. Reference implementations and the generated RoPE/QKV kernel mirror the guarded behavior while preserving required synchronization.

Changes

KV-cache slot validity

Layer / File(s)	Summary
Prefill cache-write guards models/qwen3/14b/prefill_fwd.py, models/qwen3/14b/prefill_fwd_a8w8.py, models/qwen3/14b/qwen3_14b_prefill_tq_draft.py	Prefill kernels and golden references defer cache addressing and skip K/V cache and scale writes when slots are negative.
Decode cache-write guards models/qwen3/14b/decode_layer_a8w8.py, models/qwen3/14b/qwen3_14b_decode_ssn_draft.py, models/qwen3/14b/qwen3_14b_decode_tq_draft.py	Decode paths and the TurboQuant reference guard cache addressing, cache writes, and scale writes with slot validity checks.
Generated kernel sentinel stores models/qwen3/14b/kernels/paged_attention_cce/kernel/rope_qkv_generated.hpp	Both generated KV-cache store regions now skip stores for negative sentinels while retaining synchronization operations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• hw-native-sys/pypto-lib#799: Overlaps the Qwen3 prefill RoPE/KV-cache slot-mapping logic.

Poem

I hop through slots where cache writes bloom,
Negative paths now leave more room.
K and V wait for a valid sign,
Scales follow down the guarded line.
The cache stays tidy—what a fine design!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately summarizes the main change: guarding negative slot_mapping KV writes for Qwen3-14B.
Description check	✅ Passed	The description is clearly related and matches the changeset, including the sentinel behavior and affected kernels.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request introduces a non-negative sentinel guard (slot >= 0) on slot mappings across multiple Qwen3 14B model files, including decode, prefill, and generated kernels, to prevent writing to the KV cache when the slot index is negative. The feedback highlights potential unbound variable issues in qwen3_14b_decode_ssn_draft.py and qwen3_14b_decode_tq_draft.py where block and offset variables are conditionally defined, and suggests initializing them to default values to prevent static analysis or compiler errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@models/qwen3/14b/qwen3_14b_decode_ssn_draft.py`:
- Around line 332-334: Remove the conditional guards around the block/offset
calculations in models/qwen3/14b/qwen3_14b_decode_ssn_draft.py#L332-L334 and
models/qwen3/14b/qwen3_14b_decode_tq_draft.py#L1008-L1010. Compute
rp_slot_block/rp_slot_offset and slot_block/slot_offset unconditionally, while
preserving the existing downstream guards around memory stores.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6a48f9e3-3727-4dde-affc-c2edc7fac0b5

📥 Commits

Reviewing files that changed from the base of the PR and between 0999dba and 65ae4d5.

📒 Files selected for processing (7)

• models/qwen3/14b/decode_layer_a8w8.py
• models/qwen3/14b/kernels/paged_attention_cce/kernel/rope_qkv_generated.hpp
• models/qwen3/14b/prefill_fwd.py
• models/qwen3/14b/prefill_fwd_a8w8.py
• models/qwen3/14b/qwen3_14b_decode_ssn_draft.py
• models/qwen3/14b/qwen3_14b_decode_tq_draft.py
• models/qwen3/14b/qwen3_14b_prefill_tq_draft.py

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0ec4722a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Leaf-Salix]

Update on slot_mapping=-1 / scratch-page repro

I rechecked the isolated prefill contract under the serving runtime environment.
This test only switches the pypto-lib checkout through `PYTHONPATH` / `--repo`.

Versions:

- Upstream control:
  - `hw-native-sys/pypto-lib@68889bc7e74a6e249e03758bc26d382859f74268`
- Current PR head:
  - `Leaf-Salix/pypto-lib@88b8223eacc5f5ca16ef57bc0e677e67ddf73f45`
- Previous fix point:
  - `Leaf-Salix/pypto-lib@65ae4d5681723a67d71df6ce3d693b21b3d9f859`
- Serving/runtime components:
  - `hw-native-sys/pypto-serving@db8937717193312680aa079e31c5fba9a8c2555c`
  - `hw-native-sys/pypto@2b3f017c3eed1d010bcd7301e1aeda8e2a8b2f65`
  - `hw-native-sys/simpler@a756969cfb4b25756792e6fbcb198d08beb6eea0`

Runtime environment:

```text
python: /data/sunkaixuan/yj_subdir/.venv_serving/bin/python
PTOAS_ROOT=/usr/local/bin/ptoas-bin
ASCEND_HOME_PATH=/usr/local/Ascend/cann-9.0.0
PTO_ISA_ROOT=/data/sunkaixuan/yj_subdir/pr806_repro_20260721_1918/pypto/runtime/build/pto-isa

Test parameters:

helper: run_prefill_warmup_contract_v2.py
platform: a2a3
device: 6
batch: 16
max_seq: 256
chunk_size: 256
num_layers: 40
scratch_pages: 1
slot_mode: constant
slot_value: -1
block_value: 0
PTO2_RING_HEAP=1073741824
PTO2_RING_DEP_POOL=16384
PTO2_RING_TASK_WINDOW=16384

Results:

upstream 68889bc, slot_mapping=-1, scratch_pages=1:
  FAIL_507018
  orch_error_code=2, runtime_status=-2

current PR 88b8223, slot_mapping=-1, scratch_pages=1:
  PASS

previous fix 65ae4d5, slot_mapping=-1, scratch_pages=1:
  PASS

Logs:

upstream:
  /tmp/yejia_pr806_matrix_repro_20260722_1100/logs/direct_upstream_neg1_dev6_507018.log

current PR:
  /tmp/yejia_pr806_matrix_repro_20260722_1100/logs/direct_current88_neg1_once_pass.log

Conclusion:

The original isolated `slot_mapping=-1` / scratch-page bug is still reproducible
on upstream `68889bc`, but is fixed in the current PR head `88b8223`.

The current CI failure is therefore likely not caused by the minimal negative-slot
KV-write bug still being present. It happens later in the pypto-serving
distributed prefill warmup path and should be investigated as a prefill
heap/task pressure issue.