Krea2 INT8 ConvRot Converter

A Gradio front end with two steps:

1. Dequantize FP8 -> BF16 — required if your source checkpoint is already FP8.
2. INT8 ConvRot Conversion — wraps ctq (from silveroxides/convert_to_quant), quantizing a safetensors checkpoint to INT8 with an optional Hadamard rotation (ConvRot) applied first to reduce outliers vs. plain INT8.

Setup

1. Run install.bat (creates a venv, installs PyTorch + Triton + convert_to_quant + gradio + scipy). Edit the PyTorch line in install.bat if cu128 isn't right for your GPU.
2. Run run.bat to launch the app in your browser.

Is your source checkpoint already FP8? Read this first.

If your checkpoint was ever loaded/saved through ComfyUI as FP8 (e.g. a merge made via UNETLoader with weight_dtype: fp8_e4m3fn_fast, or downloaded as an *_fp8*.safetensors file), start on the Dequantize tab — don't feed it straight into ConvRot.

Why this matters: many FP8 checkpoints store weights as raw_fp8_value plus a separate per-layer .weight_scale tensor, where the real value is raw_fp8_value.float() * weight_scale. If a quantizer doesn't know about that scale tensor and treats the raw FP8 bits as the final weight, every value it re-quantizes is off by whatever that scale factor was — across potentially hundreds of layers at once. This doesn't produce mildly-worse output; it produces total noise, because the whole model's weight magnitudes are systematically wrong before ConvRot ever runs.

Workflow:

1. Dequantize tab -> click "Scan: scaled FP8 or naive FP8?" first. It tells you exactly how many weight tensors have a companion scale tensor.
2. Click "Dequantize to BF16". This reconstructs real values using each tensor's own embedded scale (or just upcasts, for the handful of layers that don't have one — commonly the sensitive layers noted below).
3. Check the printed sample stats (abs-mean/abs-max) — these should look like ordinary weight magnitudes, not near-zero or absurdly large.
4. Take that _bf16.safetensors output into the ConvRot tab and convert that instead of the original FP8 file.

If your source checkpoint is already BF16/FP16/FP32 (never quantized), skip straight to the ConvRot tab.

ConvRot tab usage

• Prefer the local file path field over uploading. Type the real path to your checkpoint (e.g. C:\models\Soliloquy_bf16.safetensors) rather than using the upload button — avoids a Windows file-locking issue with Gradio's temp-upload handling (see Troubleshooting) and skips re-uploading a multi-GB file through the browser.
• Leave ConvRot on for the outlier-suppressed quantization; turn it off for plain INT8.
• ConvRot group size: 256 is the recommended default; pick a size that evenly divides the model's layer dimensions (64/256/1024 are supported).
• "List available presets" runs ctq --help-filters so you can see which architecture-exclusion flags your installed version supports (e.g. --flux2, --wan). There's no confirmed built-in Krea2 preset as of this writing.
• Leave "Simple mode" and "Low memory mode" on unless you specifically want the much slower SVD-based learned-rounding optimization.
• Exclude-layers field defaults to the Krea2 "K2" profile (first, last, tmlp, tproj, txtfusion, txtmlp) — these layers are documented as needing to stay unquantized for Krea2. Leave this as-is unless you know your checkpoint uses different layer names. This is a separate concern from the FP8 dequantization above — needed even on a clean BF16 source.
• "Scan checkpoint for FP8 biases" merges any additional FP8-bias layers it finds into the K2 profile rather than replacing it — safe to run before every conversion, even on an already-dequantized file.

Which output should you actually use?

BF16 will always give the best quality — it reconstructs your source as accurately as possible and runs at full precision from there. INT8 ConvRot quantizes that BF16 result a second time; ConvRot minimizes the damage from that second pass but can't fully undo it. Use INT8 ConvRot only when you need the smaller file size / faster inference and are willing to trade some fidelity for it.

Loading the outputs in ComfyUI

Both outputs now load through the standard "Load Diffusion Model" node, weight_dtype: default — no special custom node required for either variant.

(Earlier versions of this app produced files that needed a ConvRot-aware custom node like ComfyUI-INT8-Fast with model_type: krea2, because the dequantize step was leaking leftover FP8-checkpoint quantization metadata into its output — see "Leftover quantization metadata" below. That's fixed now. If you're loading a file produced by an older version of this app, or still hit a _params.clone() / 'NoneType' object has no attribute 'clone' crash on the native loader, re-run Dequantize + ConvRot with the current app.py first before reaching for a custom node.)

Troubleshooting

PermissionError when clicking Scan (file in AppData\Local\Temp\gradio\...) Windows briefly locks Gradio's temp copy of an uploaded file. Type the real path to your checkpoint into the local file path field instead of using the upload button, and click Scan again.

ModuleNotFoundError: No module named 'scipy' ConvRot needs scipy, which isn't pulled in automatically by convert_to_quant. scipy is in requirements.txt; if you hit this on an older install, run pip install scipy inside the venv.

NotImplementedError: "sum_cpu" not implemented for 'Float8_e4m3fn' Upstream ctq bug: bias-correction calls .mean() on a bias tensor still stored in FP8, and PyTorch has no CPU reduction kernel for that dtype. Fixes, in order:

1. pip install -U convert_to_quant — may already be patched upstream.
2. Dequantize to BF16 first (see above) — removes FP8 tensors from the input entirely, so this can't trigger.
3. If you still hit it, use "Scan checkpoint for FP8 biases" to auto-merge the affected layer(s) into the exclude-layers pattern.

KeyError on a *.scale tensor when loading in ComfyUI (e.g. 'txtfusion.layerwise_blocks.0.prenorm.scale') This means a real model weight (a norm layer's learnable scale/gain parameter, not a quantization scale) was dropped during dequantization. Fixed in current app.py — earlier versions matched any tensor ending in .scale/_scale as a quantization-scale companion, which deleted legitimate weights sharing that naming pattern. Re-run Dequantize with the current version.

Leftover quantization metadata — 'NoneType' object has no attribute 'clone' in ComfyUI, or console shows Found quantization metadata version 1 / Detected mixed precision quantization on what should be a plain BF16 or freshly-ConvRot'd file Fixed in current app.py. Earlier versions of Dequantize copied the source FP8 file's entire __metadata__ block and any per-tensor *.comfy_quant marker tensors straight through into the "BF16" output, even though the actual weights were correctly converted to plain BF16. ComfyUI's loader saw those leftover markers, assumed the layer was still quantized, and crashed looking for companion quantized-tensor data that no longer existed. Since that contaminated BF16 file was also what got fed into the ConvRot step, the ConvRot output inherited the same problem downstream — which is why this used to look like ConvRot INT8 specifically needed a special custom node to load. It doesn't; that was this bug. Current app.py strips quantization metadata keys and *.comfy_quant-style marker tensors during Dequantize, and its log reports exactly what was stripped so you can confirm. Re-run Dequantize (and then ConvRot from that clean output) with the current version.

Conversion seems stuck / very slow Check that PyTorch actually sees your GPU:

venv\Scripts\activate
python -c "import torch; print(torch.cuda.is_available())"

If False, ctq is running on CPU. Also make sure "Simple mode" (--simple) is checked — without it, ctq runs a much slower SVD-based learned-rounding optimization per tensor.