量化后再次压缩大模型，内存减少40%且速度几乎不变。
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
- 先重缩放参数再量化，提升权重可压缩性
- 压缩后推理内存降40%，精度和速度几乎无损
- 针对解压瓶颈提出自适应加速方法，适合移动端部署
大语言模型（LLMs）在各类任务中表现优异，但在内存受限设备上部署仍面临挑战，即使经过量化处理。本文提出一种量化后的二次压缩框架，实现约2.2倍的压缩比。首先提出一种压缩感知量化方法，通过量化前重缩放模型参数以增强权重可压缩性，随后采用剪枝进一步优化。同时发现解压过程可能成为实际应用中的性能瓶颈，因此详细分析了内存与延迟之间的权衡，并提出一种速度自适应方法予以克服。实验表明，压缩后模型推理可实现40%的内存节省，精度和推理速度几乎无损失。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit excellent performance in various tasks. However, the memory requirements of LLMs present a great challenge when deploying on memory-limited devices, even for quantized LLMs. This paper introduces a framework to compress LLM after quantization further, achieving about 2.2x compression ratio. A compression-aware quantization is first proposed to enhance model weight compressibility by re-scaling the model parameters before quantization, followed by a pruning method to improve further. Upon this, we notice that decompression can be a bottleneck during practical scenarios. We then give a detailed analysis of the trade-off between memory usage and latency brought by the proposed method. A speed-adaptive method is proposed to overcome it. The experimental results show inference with the compressed model can achieve a 40% reduction in memory size with negligible loss in accuracy and inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。