4-bit量化+QLoRA能提升大模型部署性能，兼顾效率与质量。
Can Post-Training Quantization Benefit from an Additional QLoRA Integration?
- 将4比特后训练量化与QLoRA结合，实现轻量级微调。
- 在多个数据集上超越标准量化，接近全参数微调效果。
- 适合资源受限场景下的高性能大模型部署。
大语言模型（LLMs）虽推动自然语言处理发展，但部署需大量计算资源，成本高且难获取。模型压缩技术如量化可降低资源需求，但可能损害生成质量。本研究探索4比特后训练量化（PTQ）与QLoRA的融合，通过大量实验表明，该方法优于标准PTQ，在部分情况下甚至超过16比特全参数微调，验证于私有及公开数据集，涵盖不同量化算法。结果证明了PTQ-QLoRA融合的有效性，为资源受限环境中的高性能大模型部署提供了可行方案。
原文摘要 · Abstract (English)
Large language models (LLMs) have transformed natural language processing but pose significant challenges for real-world deployment. These models necessitate considerable computing resources, which can be costly and frequently unavailable. Model compression techniques such as quantization are often leveraged to alleviate resource demand, but they may have a negative impact on the generation quality. In this study, we explore the integration of 4-bit Post-training Quantization (PTQ) with QLoRA to address these issues. We demonstrate through extensive experiments that this integration outperforms standard PTQ, and in some cases even 16-bit full-parameter fine-tuning on LLMs, validated across proprietary and public datasets with different quantization algorithms. The results demonstrate the efficacy of PTQ-QLoRA integration, offering a viable solution for deploying powerful LLMs in resource-constrained environments without compromising on performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。