arXiv:2605.10793cs.LG2026-05

通过旋转对齐激活值角落,实现低比特量化下的高效大模型推理。

ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs

论文配图:ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
图 1 · 摘自论文原文
  • 用闭式解优化正交旋转,使激活值分布更均匀。
  • 在Llama-2/3上实现3~70B模型的高精度低比特量化。
  • 无需存储激活数据或端到端训练,适合部署场景。

大语言模型因内存占用大和推理成本高而难以部署。权重与激活量化可降低开销,但低比特激活量化仍面临异常值导致的量化误差问题。现有基于旋转的方法虽能缓解此问题,但需昂贵的端到端训练或依赖存储激活数据集,带来显著计算或存储负担。本文提出一种轻量级后训练旋转校准方法:通过学习正交旋转,将归一化激活对齐至内接超立方体的角点,促使激活能量更均匀分布于各维度。该目标可通过正交Procrustes问题的闭式解高效更新,避免在正交群上进行梯度优化。进一步设计在线校准流程,在处理校准样本时实时更新旋转参数,无需将激活数据存盘,并支持旋转随量化激活分布自适应调整。在3B至70B参数的Llama-2和Llama-3模型上实验表明,本方法在困惑度和常识推理任务中达到竞争力或更优性能,同时规避了高昂的端到端训练与大规模离线激活存储。

原文摘要 · Abstract (English)

Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost. Weight-activation quantization can reduce these costs, but low-bit activation quantization remains difficult because activation outliers induce large quantization error. Recent rotation-based methods address this by applying orthogonal transformations that redistribute activation magnitude across dimensions, but existing approaches either require expensive end-to-end rotation training or rely on stored activation corpora, introducing significant compute or storage overhead. We propose a lightweight post-training rotation calibration method for LLM activation quantization. Our method learns orthogonal rotations that align normalized activations with the corners of an inscribed hypercube, encouraging activation energy to be distributed more evenly across dimensions. This objective admits an efficient closed-form update via the orthogonal Procrustes problem, avoiding gradient-based optimization over the orthogonal group. We further introduce an online calibration procedure that updates rotations as calibration samples are processed, eliminating the need to store activations on disk and allowing rotations to adapt to quantized activation distributions during calibration. Experiments on Llama-2 and Llama-3 models from 3B to 70B parameters show that our method achieves competitive or improved performance across perplexity benchmarks and common sense reasoning tasks while avoiding both costly end-to-end training and large offline activation storage.

量化大模型旋转校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。