arXiv:2505.17595cs.LGcs.CL2025-05中稿 · ICML被引 1

提出新方法优化低比特大模型量化初始参数,提升性能且部署更高效。

NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs

  • 通过解析零点简化为仅优化尺度的策略,实现高效初始化。
  • 在LLaMA和Qwen模型上均显著优于现有方法,部分任务超越复杂调优。
  • 适合追求高效部署与高性能的低比特大模型应用开发者。

大语言模型(LLMs)虽在多领域表现优异,但因内存占用高、推理成本大,在消费级GPU或个人设备(如笔记本)上部署困难。后训练量化(PTQ)可有效降低内存占用和解码延迟。实践中,统一量化因效率高且被主流软硬件广泛支持而被优先采用。尽管低比特统一量化近年取得进展,但研究多集中于量化方法本身,量化参数初始化仍依赖传统最小-最大公式,未受充分重视。本文识别出该公式的局限性,提出NeUQI方法,高效确定近似最优的统一量化初始参数。通过推导给定尺度下的零点,将联合优化问题简化为仅尺度优化,显著提升初始化质量。实验表明,NeUQI在LLaMA和Qwen系列模型上多种设置与任务中均持续优于现有方法;结合轻量级知识蒸馏后,性能甚至超过资源消耗更高的PV-tuning。

原文摘要 · Abstract (English)

Large language models (LLMs) achieve impressive performance across domains but face significant challenges when deployed on consumer-grade GPUs or personal devices such as laptops, due to high memory consumption and inference costs. Post-training quantization (PTQ) of LLMs offers a promising solution that reduces their memory footprint and decoding latency. In practice, PTQ with uniform quantization representation is favored due to its efficiency and ease of deployment, as uniform quantization is widely supported by mainstream hardware and software libraries. Recent studies on low-bit uniform quantization have led to noticeable improvements in post-quantization model performance; however, they mainly focus on quantization methodologies, while the initialization of quantization parameters remains underexplored and still relies on the conventional Min-Max formula. In this work, we identify the limitations of the Min-Max formula, move beyond its constraints, and propose NeUQI, a method that efficiently determines near-optimal initialization for uniform quantization. Our NeUQI simplifies the joint optimization of the scale and zero-point by deriving the zero-point for a given scale, thereby reducing the problem to a scale-only optimization. Benefiting from the improved quantization parameters, our NeUQI consistently outperforms existing methods in the experiments with the LLaMA and Qwen families on various settings and tasks. Furthermore, when combined with a lightweight distillation strategy, NeUQI even achieves superior performance to PV-tuning, a considerably more resource-intensive method.

大模型量化统一量化低比特推理模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。