一拍即合的多比特量化，无需重复训练
QuEPT: Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching
- 单次校准重构块级多比特误差，动态适配不同精度
- 支持实时切换均匀与混合精度，精度接近最优方法
- 适合部署在资源受限的LLM场景，兼顾效率与性能
弹性精度量化通过一次优化实现多比特部署，适应多样化量化需求。然而，由于Transformer架构带来的高存储与优化成本，针对大语言模型的弹性量化研究仍有限。本文提出QuEPT，一种高效的后训练方案，仅需少量数据即可完成块级多比特误差的一次性校准重建。通过级联不同低秩适配器，可动态适配预设的多种位宽，并支持在统一量化与混合精度之间实时切换，无需重复优化。为提升精度与鲁棒性，引入多比特令牌融合（MB-ToMe），动态融合不同位宽下的特征表示，增强切换时的稳定性；同时提出多比特级联低秩适配器（MB-CLoRA），强化位宽组间的关联性，进一步提升整体性能。大量实验表明，QuEPT在性能上达到或超越现有先进后训练量化方法。代码已开源：https://github.com/xuke225/QuEPT。
原文摘要 · Abstract (English)
Elastic precision quantization enables multi-bit deployment via a single optimization pass, fitting diverse quantization scenarios.Yet, the high storage and optimization costs associated with the Transformer architecture, research on elastic quantization remains limited, particularly for large language models.This paper proposes QuEPT, an efficient post-training scheme that reconstructs block-wise multi-bit errors with one-shot calibration on a small data slice. It can dynamically adapt to various predefined bit-widths by cascading different low-rank adapters, and supports real-time switching between uniform quantization and mixed precision quantization without repeated optimization. To enhance accuracy and robustness, we introduce Multi-Bit Token Merging (MB-ToMe) to dynamically fuse token features across different bit-widths, improving robustness during bit-width switching. Additionally, we propose Multi-Bit Cascaded Low-Rank adapters (MB-CLoRA) to strengthen correlations between bit-width groups, further improve the overall performance of QuEPT. Extensive experiments demonstrate that QuEPT achieves comparable or better performance to existing state-of-the-art post-training quantization methods.Our code is available at https://github.com/xuke225/QuEPT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。