RaanA让大模型量化更快更省数据，还能自由分配精度。
RaanA: A Fast, Flexible, and Data-Efficient Post-Training Quantization Algorithm
- 用随机向量量化+分层敏感度分配比特，实现快速高效量化
- 仅需少量校准数据即可达到顶尖性能，支持灵活比特配置
- 适合追求低延迟、低资源部署的模型优化工程师
后训练量化（PTQ）已成为提升大语言模型推理效率的常用技术。然而，现有方法普遍存在校准数据需求高、目标比特数选择僵化等关键缺陷。本文提出RaanA，一种统一的PTQ框架，通过引入两个新组件克服上述挑战：1）RaBitQ-H，一种改进的随机向量量化方法，实现快速、精准且高效的量化；2）AllocateBits，根据各层量化敏感度最优分配比特宽度。RaanA在保持与当前最先进量化方法相当性能的同时，速度极快，所需校准数据极少，并支持灵活的比特分配。大量实验验证了RaanA在效率与精度间的良好平衡。代码已公开于 https://github.com/FFTYYY/RaanA。
原文摘要 · Abstract (English)
Post-training Quantization (PTQ) has become a widely used technique for improving inference efficiency of large language models (LLMs). However, existing PTQ methods generally suffer from crucial limitations such as heavy calibration data requirements and inflexible choice of target number of bits. In this paper, we propose RaanA, a unified PTQ framework that overcomes these challenges by introducing two novel components: 1) RaBitQ-H, a variant of a randomized vector quantization method RaBitQ, designed for fast, accurate, and highly efficient quantization; and 2) AllocateBits, an algorithm that optimally allocates bit-widths across layers based on their quantization sensitivity. RaanA achieves competitive performance with state-of-the-art quantization methods while being extremely fast, requiring minimal calibration data, and enabling flexible bit allocation. Extensive experiments demonstrate RaanA's efficacy in balancing efficiency and accuracy. The code is publicly available at https://github.com/FFTYYY/RaanA .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。