通过延迟与可调性信息精准剪枝，提升大模型推理速度与准确率。
Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information
- 结合延迟收益与可调性，智能选择需剪枝的子层。
- 在零样本常识推理上比现有方法高23.88%准确率。
- 适合追求高速高准大模型部署的研究者与工程师。
如何在不牺牲准确率的前提下加速大语言模型（LLMs）？LLMs 推理速度慢限制了其在多样化应用中的实际收益，主要因大量子层堆叠所致。子层剪枝通过移除冗余子层来压缩并加速模型。然而，现有剪枝算法因盲目选择剪枝对象而准确率受限，忽视了各子层的差异特性。本文提出 SPRINT（Sublayer PRuning wIth LateNcy and Tunability Information），一种基于延迟与可调性信息的精准子层剪枝方法。SPRINT 通过评估剪枝后的延迟降低量与子层可调性，迭代地剪除冗余子层，并快速微调剩余子层参数。实验表明，SPRINT 在准确率-加速比权衡上表现最优，在零样本常识推理基准上相比现有剪枝算法最高提升 23.88% 的准确率。
原文摘要 · Abstract (English)
How can we accelerate large language models(LLMs) without sacrificing accuracy? The slow inference speed of LLMs hinders us to benefit from their remarkable performance in diverse applications. This is mainly because numerous sublayers are stacked together in LLMs. Sublayer pruning compresses and expedites LLMs via removing unnecessary sublayers. However, existing sublayer pruning algorithms are limited in accuracy since they naively select sublayers to prune, overlooking the different characteristics of each sublayer. In this paper, we propose SPRINT (Sublayer PRuning wIth LateNcy and Tunability Information), an accurate sublayer pruning method for LLMs. SPRINT accurately selects a target sublayer to prune by considering 1) the amount of latency reduction after pruning and 2) the tunability of sublayers. SPRINT iteratively prunes redundant sublayers and swiftly tunes the parameters of remaining sublayers. Experiments show that SPRINT achieves the best accuracy-speedup trade-off, exhibiting up to 23.88%p higher accuracy on zero-shot commonsense reasoning benchmarks compared to existing pruning algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。