提出细粒度投影剪枝，让大模型更省资源且保持高精度。
Mosaic: Composite Projection Pruning for Resource-efficient LLMs
- 用投影剪枝实现细粒度压缩，结合结构与非结构剪枝优势。
- 比传统方法快7.19倍，推理速度提升67%，显存降低68%。
- 适合需要高效部署大模型的工程师和研究者使用。
大规模语言模型（LLMs）因计算和内存需求高，难以在各类硬件上部署。压缩技术如剪枝可降低模型规模，从而减少资源消耗。现有剪枝方法多为粗粒度，耗时长且易损失关键参数，影响模型性能。本文提出投影剪枝这一新型细粒度剪枝方法，并进一步引入复合投影剪枝——将保留精度的非结构化剪枝与减小模型尺寸的结构化剪枝协同结合。我们开发了Mosaic系统，用于生成并部署采用复合投影剪枝的压缩模型。在多种硬件平台、大模型及数据集上评估显示，Mosaic生成模型的速度比现有方法快7.19倍；相比粗粒度剪枝，其困惑度降低84.2%，准确率提升31.4%；推理速度最高提升67%，GPU显存占用减少68%。Mosaic已开源，可通过https://github.com/blessonvar/Mosaic获取。
原文摘要 · Abstract (English)
Extensive compute and memory requirements limit the deployment of large language models (LLMs) on any hardware. Compression methods, such as pruning, can reduce model size, which in turn reduces resource requirements. State-of-the-art pruning is based on coarse-grained methods. They are time-consuming and inherently remove critical model parameters, adversely impacting the quality of the pruned model. This paper introduces projection pruning, a novel fine-grained method for pruning LLMs. In addition, LLM projection pruning is enhanced by a new approach we refer to as composite projection pruning - the synergistic combination of unstructured pruning that retains accuracy and structured pruning that reduces model size. We develop Mosaic, a novel system to create and deploy pruned LLMs using composite projection pruning. Mosaic is evaluated using a range of performance and quality metrics on multiple hardware platforms, LLMs, and datasets. Mosaic is 7.19x faster in producing models than existing approaches. Mosaic models achieve up to 84.2% lower perplexity and 31.4% higher accuracy than models obtained from coarse-grained pruning. Up to 67% faster inference and 68% lower GPU memory use is noted for Mosaic models. Mosaic is available for public use from https://github.com/blessonvar/Mosaic
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。