Skip to content

Fix Flash-Next FP8 n-gram scalar scales - #474

Open
GrahamJenkins wants to merge 1 commit into
youssofal:mainfrom
GrahamJenkins:fix/qwen38-flash-next-fp8-ngram-scale
Open

Fix Flash-Next FP8 n-gram scalar scales#474
GrahamJenkins wants to merge 1 commit into
youssofal:mainfrom
GrahamJenkins:fix/qwen38-flash-next-fp8-ngram-scale

Conversation

@GrahamJenkins

Copy link
Copy Markdown

Summary

The stock converter fails on the official Qwen/Qwen3.8-Flash-Next-FP8 revision 236dfdf285828023ca3bcd3f37366c58a3469b13 because its 128 PLE n-gram FP8 shards share one BF16 scalar weight_scale rather than providing per-weight weight_scale_inv tensors.

This narrow fix falls back to that shared scalar only for n-gram shards, broadcasts it during FP8 dequantization, and accounts for the shared source tensor in fail-closed accounting. Ordinary block-scale weight_scale_inv conversion remains unchanged.

Validation

  • Added synthetic regression coverage for both ordinary 128x128 block weight_scale_inv and official n-gram shared scalar weight_scale layouts.
  • Targeted regression tests pass on Linux with a NumPy-backed MLX stand-in; native MLX execution is not available on this host.
  • Completed the full 185.6 GB official-source conversion, Forge verification, and model-load verification on Nexus. The verified patched converter SHA-256 is 4ecc6eb37366739ba214778adea7ba6aeacf133817d5162cba684abc8f1a8b73.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant