LoRAQuant通过混合精度量化，让LoRA适配器压缩到极低比特仍保持高性能。
LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
- 用SVD重参数化，将关键信息集中到特定行列，便于差异化量化。
- 在多个大模型上测试，比特数远低于现有方法，性能却相当或更优。
- 适合需要同时加载多个适配器的个性化或多任务场景。
低秩适配（LoRA）已成为大语言模型参数高效微调的流行技术。在实际应用中，常需同时加载多个适配器以实现个性化体验或支持多样化任务。尽管单个适配器轻量，但其累积开销在大规模下显著。为此，我们提出专为LoRA设计的混合精度后训练量化方法LoRAQuant。具体而言，LoRAQuant通过奇异值分解（SVD）对每个适配器进行重参数化，将最重要信息集中于特定行和列，从而可对关键部分采用高精度量化，其余部分则压缩至超低比特。我们在LLaMA 2-7B、LLaMA 2-13B及Mistral 7B模型上针对数学推理、编码和摘要任务进行了全面实验。结果表明，相比其他量化方法，LoRAQuant使用显著更低的比特数，但仍能达到相当甚至更高的性能。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs). In many real-world scenarios, multiple adapters are loaded simultaneously to enable LLM customization for personalized user experiences or to support a diverse range of tasks. Although each adapter is lightweight in isolation, their aggregate cost becomes substantial at scale. To address this, we propose LoRAQuant, a mixed-precision post-training quantization method tailored to LoRA. Specifically, LoRAQuant reparameterizes each adapter by singular value decomposition (SVD) to concentrate the most important information into specific rows and columns. This makes it possible to quantize the important components to higher precision, while quantizing the rest to ultra-low bitwidth. We conduct comprehensive experiments with LLaMA 2-7B, LLaMA 2-13B, and Mistral 7B models on mathematical reasoning, coding, and summarization tasks. Results show that our LoRAQuant uses significantly lower bits than other quantization methods, but achieves comparable or even higher performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。