提出RILQ方法，提升2比特大模型推理精度
RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracy
- 利用激活差异损失的秩无关特性，协同调整各层适配器
- 在LLaMA-2/3上实现2比特量化下精度显著提升
- 保持低秩适配效率，适合轻量化部署场景
低秩适配（LoRA）已成为高效微调大语言模型的主流方法，基于LoRA的量化误差补偿（LQEC）也成为恢复压缩后大模型精度的重要工具。然而，现有LQEC在4比特以下场景表现不佳，且缺乏对其局限性的系统研究。本文提出RILQ（秩无关的LoRA量化误差补偿），通过分析发现模型级激活差异损失具有秩无关特性，据此设计跨层协同调整的适配器更新机制，实现鲁棒的误差补偿。在LLaMA-2和LLaMA-3上的实验表明，RILQ在多种先进量化器下均能持续提升2比特量化推理精度，并在任务特定微调中表现更优。RILQ保持与现有LoRA方法相当的计算效率，支持适配器合并后的权重量化推理，显著提升2比特大模型性能，是推动低比特大模型实用化的有效方案。代码已开源。
原文摘要 · Abstract (English)
Low-rank adaptation (LoRA) has become the dominant method for parameter-efficient LLM fine-tuning, with LoRA-based quantization error compensation (LQEC) emerging as a powerful tool for recovering accuracy in compressed LLMs. However, LQEC has underperformed in sub-4-bit scenarios, with no prior investigation into understanding this limitation. We propose RILQ (Rank-Insensitive LoRA-based Quantization Error Compensation) to understand fundamental limitation and boost 2-bit LLM accuracy. Based on rank analysis revealing model-wise activation discrepancy loss's rank-insensitive nature, RILQ employs this loss to adjust adapters cooperatively across layers, enabling robust error compensation with low-rank adapters. Evaluations on LLaMA-2 and LLaMA-3 demonstrate RILQ's consistent improvements in 2-bit quantized inference across various state-of-the-art quantizers and enhanced accuracy in task-specific fine-tuning. RILQ maintains computational efficiency comparable to existing LoRA methods, enabling adapter-merged weight-quantized LLM inference with significantly enhanced accuracy, making it a promising approach for boosting 2-bit LLM performance. Our code is available at https://github.com/aiha-lab/RILQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。