用谱能量引导量化，让低比特大模型更准
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
- 按沃尔什基能量重排权重，聚焦重要通道
- 在1.5B模型上困惑度降低58%，显著优于基线
- 适配多种架构，可直接部署于英特尔硬件
我们将一篇配套理论论文（arXiv:2605.01637）中的影响自适应沃尔什几何应用于极端低比特仅权重大模型量化。方法为单一数学不变变换：对每一层权重矩阵进行WHT旋转，并按坐标级沃尔什基激活能量缩放列向量，再交由重构误差量化器（Intel auto-round）处理。该策略使每组整数舍入偏向高谱能量通道。在4个预训练解码器模型（参数规模135M至1.5B）上，相较于原生auto-round，BBT-谱方法在W2A16设置下使WikiText-2困惑度降低15%-58%；还报告了TinyLlama-1.1B的辅助数据点。三项扩展将该方法推广至原不适用的模型族：对Qwen3注意力使用逐头PCA矩阵-Gamma替换q_norm/k_norm（Qwen3-0.6B上PPL从136.76降至88.99）；引入与RoPE可交换的SO(2)成对旋转（Qwen2.5-1.5B上PPL从36.93降至21.84）；通过架构模糊测试发现的针对Laguna型融合专家结构的输入侧吸收修正。W2与W4的对比实验显示，当比特降至W4时，再分配收益落入±0.5 PPL噪声范围，符合舒尔凸性直觉——当噪声预算缩小时，非集中影响的成本趋于消失。所有量化权重可导出为OpenVINO IR，在Intel NPU、Arc dGPU和CPU上运行，困惑度设备间差异小于±0.1。本文未宣称形式上将理论论文的主序论证从布尔值转移到实值：此处使用的WHT激活能量并非理论论文中的布尔影响，关联为直观理解，贡献在于工程价值而非定理传递。与SpinQuant、QuaRot、QuIP-sharp、AQLM、OmniQuant、ButterflyQuant在相同校准下的直接对比是主要未来工作。
原文摘要 · Abstract (English)
We apply the influence-adaptive Walsh geometry of a companion theory paper (arXiv:2605.01637) to extreme low-bit weight-only LLM quantization. The recipe is one math-invariant transformation: WHT-rotate each linear layer's weight matrix and rescale its columns by per-coordinate Walsh-basis activation energy before handing off to a reconstruction-error quantizer (Intel auto-round). This biases per-group integer rounding toward high-spectral-energy channels. On four pretrained decoder-only models from 135M to 1.5B parameters, BBT-spectral reduces wikitext-2 perplexity by 15-58% relative to vanilla auto-round at W2A16; we also report a TinyLlama-1.1B auxiliary data point. Three extensions transfer the recipe to families it failed on: a per-head PCA matrix-Gamma replacement of q_norm/k_norm for Qwen3 attention (PPL 136.76 -> 88.99 on Qwen3-0.6B); an SO(2) per-pair rotation that commutes with RoPE (PPL 36.93 -> 21.84 on Qwen2.5-1.5B); and an MoE-aware input-side absorption fix identified by architectural fuzzing of Laguna-style fused-expert layouts. A W2-vs-W4 ablation gives a deliberate negative control: the redistribution payoff falls within the +/-0.5 PPL noise floor at W4, consistent with the Schur-convexity intuition that the cost of unconcentrated influence vanishes as the noise budget shrinks. All quantized weights export to OpenVINO IR and run on Intel NPU + Arc dGPU + CPU with PPL invariant to device within +/-0.1. We do not claim a formal Boolean-to-real-valued transfer of the theory paper's majorization argument: the WHT activation energy used here is not the Boolean influence of the theory paper, the link is intuitive, and the contribution is engineering value rather than a transferred theorem. Head-to-head benchmarks against SpinQuant, QuaRot, QuIP-sharp, AQLM, OmniQuant, and ButterflyQuant at matched calibration are the main future-work item.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。