针对大模型推理压缩难题，提出细粒度低秩压缩方法
FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
- 按层动态分配压缩率，避免统一压缩导致性能下降
- 引入渐进式解码机制，在压缩下保持生成质量
- 在摘要任务上比现有方法提升17% ROUGE-L
尽管大语言模型（LLM）取得了显著性能，但其庞大的参数量阻碍了在资源受限设备上的部署。低秩压缩可降低内存占用和计算需求，但对所有层采用统一压缩率常导致性能显著下降，且以往方法在解码阶段表现不佳。为此，本文提出细粒度低秩压缩器（FLRC），能为每层高效确定最优压缩秩，并引入渐进式低秩解码以维持文本生成质量。在多种基准上的全面实验表明，FLRC性能优越，在摘要任务上相比当前最优的低秩压缩方法，ROUGE-L最高提升17%，构建了更鲁棒高效的LLM推理框架。
原文摘要 · Abstract (English)
Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression can reduce both memory usage and computational demand, but applying a uniform compression ratio across all layers often leads to significant performance degradation, and previous methods perform poorly during decoding. To address these issues, we propose the Fine-grained Low-Rank Compressor (FLRC), which efficiently determines an optimal rank allocation for each layer, and incorporates progressive low-rank decoding to maintain text generation quality. Comprehensive experiments on diverse benchmarks demonstrate the superiority of FLRC, achieving up to a 17% improvement in ROUGE-L on summarization tasks compared to state-of-the-art low-rank compression methods, establishing a more robust and efficient framework to improve LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。