让大模型更快更省资源，用量化技术降低使用门槛
Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
- 提出训练后量化方法，压缩模型体积提升推理速度
- 支持多种精度方案与粒度选择，平衡性能与效率
- 适合想低成本部署大模型的开发者和研究者
大型语言模型在自然语言处理中取得显著进展，但其高昂的资源需求给硬件可及性和能耗带来严峻挑战。本文系统综述了面向终端用户的训练后量化（PTQ）技术，涵盖多种量化方案、粒度级别及权衡关系，旨在为该技术的理论与应用提供平衡且全面的视角。
原文摘要 · Abstract (English)
Large language models have significantly advanced natural language processing, yet their heavy resource demands pose severe challenges regarding hardware accessibility and energy consumption. This paper presents a focused and high-level review of post-training quantization (PTQ) techniques designed to optimize the inference efficiency of LLMs by the end-user, including details on various quantization schemes, granularities, and trade-offs. The aim is to provide a balanced overview between the theory and applications of post-training quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。