2比特量化下提升大模型性能，无需训练即可优化旋转变换
Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
- 用序数排列的沃尔什-哈达玛矩阵构造旋转矩阵，降低量化误差
- 在WikiText-2上实现接近优化方法的困惑度表现，2比特仍有效
- 可兼容已有旋转方法，适合部署低比特大模型的研究者
大语言模型因计算开销大难以部署，后训练量化（PTQ）虽能缓解但现有基于旋转的方法在2比特等极低位宽下表现不佳。本文提出一种无训练的改进旋转矩阵构造方法，利用具有序数排序的沃尔什-哈达玛变换，将相似频率成分聚集，减少量化误差。进一步提出分组序数排列旋转（GSR），采用小块对角矩阵结构，有效隔离异常值影响，在不需训练的情况下达到与优化方法相当的性能。该方法在推理任务和WikiText-2上的困惑度（PPL）表现稳健，且可增强已有学习型旋转技术的效果。
原文摘要 · Abstract (English)
Large Language Models (LLMs) face deployment challenges due to high computational costs, and while Post-Training Quantization (PTQ) offers a solution, existing rotation-based methods struggle at very low bit-widths like 2-bit. We introduce a novel, training-free approach to construct an improved rotation matrix, addressing the limitations of current methods. The key contributions include leveraging the Walsh-Hadamard transform with sequency ordering, which clusters similar frequency components to reduce quantization error compared to standard Hadamard matrices, significantly improving performance. Furthermore, we propose a Grouped Sequency-arranged Rotation (GSR) using block-diagonal matrices with smaller Walsh blocks, effectively isolating outlier impacts and achieving performance comparable to optimization-based methods without requiring any training. Our method demonstrates robust performance on reasoning tasks and Perplexity (PPL) score on WikiText-2. Our method also enhances results even when applied over existing learned rotation techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。