alt.hn

8/3/2026 at 4:07:09 PM

Explanation of INT8 ConvRot (FP8 is no longer needed)

https://note.com/hirorohi03/n/n047a8c5f7f8b?hl=en

by peter_d_sherman

8/3/2026 at 7:56:36 PM

The other CloudFlare post on the front page has an interesting passage in it

> It is compute-bound, and INT4 weights have to be expanded back out before the model can multiply with them, so that extra step makes prefill slower rather than faster, GLM sustains about 10,160 tokens per second of prefill in FP8 versus 8,660 in INT4. As with the KV cache, the disaggregated design turns this into a choice rather than a compromise: we run INT4 for decode, where it wins, and FP8 for prefill, where it wins.

So each of these improvements are useful even if they have a narrow area of applicability, since the systems can be hybridized for performance.

[1] - https://blog.cloudflare.com/smaller-faster-safer-models/

by gopalv

8/3/2026 at 8:44:08 PM

Is there any degradation with INT8 weights quantization? Why would anyone want to apply ConvRot to do 8 bit weights? Note the paper [1] focuses on 4 bit weights and 4 bit activations (W4A4) quant scheme - a much more challenging goal. My understanding is that W8A8 with weights quantized per-output-channel is a solved problem - does not need any fancy methods like GPTQ, SpinQuant etc.

[1] https://arxiv.org/abs/2512.03673

by p1esk

8/3/2026 at 6:47:39 PM

What about AMD? I'm guessing it's not supported, which is a shame, because they're better value for VRAM.

by randomblock1

8/3/2026 at 5:08:30 PM

One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the benefit for prefill only, while maintaining the same decoding performance.

by liuliu

8/3/2026 at 5:08:09 PM

Alternative title: "Our invention makes INT8 better on older consumer cards without FP8 support"

by augment_me

8/3/2026 at 4:17:49 PM

>"FP8, which has been used as the standard until now, was only supported by the RTX 40/50 series. In contrast,

INT8 ConvRot

is attracting attention because it can benefit from hardware support even on the RTX 20/30 series."

https://blog.comfy.org/p/minimax-h3-day-0-support-in-comfyui

by peter_d_sherman

8/3/2026 at 4:27:15 PM

Just read that before finding this. Makes me feel like we're all on the same brain-wavelength. Im actually in the middle of pulling and setting up H3 locally (int8 convrot)

by spottedmarley