2bit量化让大模型失效？论文发现两种截然不同的崩溃原因。
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
- 区分出信号退化和计算坍塌两类失败机制
- 2比特量化下性能暴跌，因早期层信号彻底丢失
- 修复信号退化可行，但计算坍塌需重构模型结构
后训练量化（PTQ）对大语言模型高效部署至关重要。尽管4比特量化被视为最佳权衡，但降至2比特通常引发灾难性性能下降。其根本机制是否不同尚不明确。为此，我们开展系统性机理分析，揭示两类质异的失败模式：信号退化指计算模式保留但信息精度受累积误差影响；计算坍塌则导致关键组件失效，早期层即破坏信号，无法正确处理信息。基于此诊断，我们实施机制感知干预，证明针对性、无需训练的修复可缓解信号退化，但对计算坍塌无效。研究提供一套系统的PTQ故障诊断框架，表明应对计算坍塌需结构重构而非简单补偿。
原文摘要 · Abstract (English)
Post-Training Quantization (PTQ) is critical for the efficient deployment of Large Language Models (LLMs). While 4-bit quantization is widely regarded as an optimal trade-off, reducing the precision to 2-bit usually triggers a catastrophic ``performance cliff.'' It remains unclear whether the underlying mechanisms differ fundamentally. Consequently, we conduct a systematic mechanistic analysis, revealing two qualitatively distinct failure modes: Signal Degradation, where the computational patterns remain intact but information precision is impaired by cumulative error; and Computation Collapse, where key components fail to function, preventing correct information processing and destroying the signal in the early layers. Guided by this diagnosis, we conduct mechanism-aware interventions, demonstrating that targeted, training-free repair can mitigate Signal Degradation, but remains ineffective for Computation Collapse. Our findings provide a systematic diagnostic framework for PTQ failures and suggest that addressing Computation Collapse requires structural reconstruction rather than mere compensation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。