4-bit量化让混合模型的递归层稳定运行，关键在误差控制机制。
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
- 用块级缩放的NVFP4量化所有线性层，包括递归模块GDN
- 17.5GiB模型在多个任务上接近全精度表现，32K上下文误差随位置缩小
- 适合追求极致压缩与推理速度的部署场景
混合大模型将softmax注意力与线性注意力层（如Gated DeltaNet, GDN）结合，其递归状态以固定大小总结上下文。早期4比特量化对Qwen3.8-27B（48层GDN，16层注意力）仅保留GDN模块为8或16位精度，基于递归误差随长上下文累积的直觉。我们通过构建Minima：在全部496个线性层（含GDN）上采用NVFP4 W4A4，验证该直觉。在4K/32K上下文困惑度、MMLU-Pro、GSM8K、AIME'25、GPQA-Diamond、LiveCodeBench及RULER检索至64K等任务中，Minima表现与BF16相当（5项平均-0.52），体积最小（17.5 GiB），预填充速度提升14%-19%。四部分机制分析揭示原因：(i) NVFP4的16元素块缩放将残差流极端异常值局部化，均衡各层激活误差；(ii) 本被认为脆弱的门控投影实际最不敏感——softplus/指数与sigmoid参数化将约11%的GEMM误差压缩至约2%输出误差；(iii) delta规则递归使注入噪声在32K token内保持平坦平台，并在数百步内遗忘状态脉冲，因每次写入沿当前键方向覆盖状态；(iv) 每token量化代价随上下文增加而稀释，而非累积。此外修复了模块校准后融合成单个GEMM时的全局失配问题，证明校准后的FP8 KV缓存无性能损失。结果：提出可实践方案——全量量化+传输KV尺度——并从机理上解释为何混合模型的递归部分是易量化部分。检查点：https://huggingface.co/minima-ai/mnma_qwen3.8_27b_nvfp4
原文摘要 · Abstract (English)
Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attention layers) left the GDN block in 8- or 16-bit precision -- especially its decay and write-strength gates -- on the intuition that errors in a recurrence accumulate over long contexts. We test that intuition by building Minima: NVFP4 W4A4 on all 496 linear layers, GDN included. Across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K, Minima matches BF16 within seed noise (5-task average -0.52) while being the smallest (17.5 GiB) and fastest-prefill (+14-19%) recipe we compare, and its 32K perplexity gap shrinks with position. A four-part mechanism study explains why: (i) NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; (ii) the supposedly fragile gate projections are the least sensitive -- softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error; (iii) the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens and forgets a state impulse within hundreds of steps, because each write overwrites the state along the current key direction; (iv) the per-token quantization cost washes out with context instead of compounding. We also repair a global-scale mismatch that arises when per-module-calibrated NVFP4 checkpoints are served by kernels that fuse those modules into one GEMM, and show calibrated FP8 KV-cache scales are performance-free. The result: a practical recipe -- quantize everything, ship KV scales -- and a mechanistic account of why the recurrent half of a hybrid LLM is the easy half to quantize. Checkpoint: https://huggingface.co/minima-ai/mnma_qwen3.8_27b_nvfp4
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。