8位量化+早退机制让大模型更快更准，哈达玛变换反而是性能杀手
BitSkip: An Empirical Analysis of Quantization and Early Exit Composition in Transformers
- 用8位量化结合早退设计，提升推理效率
- 8位模型比4位和哈达玛增强版更好，仅4%损失换32.5%提速
- 哈达玛变换导致性能暴跌超37000%，训练极不稳定
为提升大语言模型的效率，研究者采用极端量化与动态路由等复杂技术。然而这些方法组合后的效果仍不明确。本文提出BitSkip框架系统分析其交互作用。出人意料的是，不带哈达玛变换的8位量化模型（BitSkip-V1）不仅优于4位及哈达玛增强版本，且在困惑度上达到1.13，接近全精度基线（1.19）。引入哈达玛变换后，即使在8位精度下，性能也下降超过37000%，源于根本性训练不稳定性。BitSkip-V1展现出优异早退特性：第18层退出可实现32.5%加速，质量仅损失4%。
原文摘要 · Abstract (English)
The pursuit of efficient Large Language Models (LLMs) has led to increasingly complex techniques like extreme quantization and dynamic routing. While individual benefits of these methods are well-documented, their compositional effects remain poorly understood. This paper introduces BitSkip, a hybrid architectural framework for systematically exploring these interactions. Counter-intuitively, our findings reveal that a simple 8-bit quantized model without Hadamard transform (BitSkip-V1) not only outperforms its more complex 4-bit and Hadamard-enhanced counterparts but also competes the full-precision baseline in quality (perplexity of 1.13 vs 1.19) . The introduction of Hadamard transforms, even at 8-bit precision, catastrophically degraded performance by over 37,000%, tracing fundamental training instability. Our BitSkip-V1 recipe demonstrates superior early-exit characteristics, with layer 18 providing optimal 32.5% speed gain for minimal 4% quality loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。