根据输入动态选择模型层，让大模型推理更省资源。
IG-Pruning: Input-Guided Block Pruning for Large Language Models
- 基于输入语义自动挑选该用哪些模型层
- 比固定剪枝方法在多种任务上表现更好
- 无需额外训练，适合设备资源有限的场景
随着大语言模型计算需求的增长，高效推理对实际部署愈发关键。深度剪枝通过移除Transformer层来降低计算成本，成为有前景的方法。然而，现有方法通常使用固定的层掩码，导致在不同任务和输入下性能不佳。本文提出IG-Pruning，一种新型的输入感知分块剪枝方法，在推理时动态选择层掩码。方法包含两个阶段：(1) 通过语义聚类与L0优化发现多样化的掩码候选；(2) 实现无需大量训练的高效动态剪枝。实验表明，该方法持续优于当前最先进的静态深度剪枝方法，特别适合资源受限的部署场景。
原文摘要 · Abstract (English)
With the growing computational demands of large language models (LLMs), efficient inference has become increasingly critical for practical deployment. Depth pruning has emerged as a promising approach for reducing the computational costs of large language models by removing transformer layers. However, existing methods typically rely on fixed block masks, which can lead to suboptimal performance across different tasks and inputs. In this paper, we propose IG-Pruning, a novel input-aware block-wise pruning method that dynamically selects layer masks at inference time. Our approach consists of two stages: (1) Discovering diverse mask candidates through semantic clustering and L0 optimization, and (2) Implementing efficient dynamic pruning without the need for extensive training. Experimental results demonstrate that our method consistently outperforms state-of-the-art static depth pruning methods, making it particularly suitable for resource-constrained deployment scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。