针对大模型推理延迟与体积,提出双阶段结构化剪枝方法。
Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization

- 分两阶段剪枝:先删整块注意力与MLP,再用贝叶斯优化精准调参
- 在37.5%和50%剪枝率下,零样本推理性能优于现有方法
- 兼顾延迟、大小与精度,适合边缘设备部署
大型语言模型(LLMs)凭借强大的推理与问答能力获得广泛应用。然而,在嵌入式和边缘计算环境中部署仍面临严格延迟、内存和能耗限制。其庞大的参数量与计算需求阻碍了在资源受限平台上的高效运行。尽管模型剪枝成为缩小规模的可行方案,但同时优化层、注意力头与多层感知机(MLP)维度仍极复杂。全面探索这一联合设计空间计算成本高,常陷入局部最优或配置不稳定。为此,我们提出一种硬件感知的多目标结构化剪枝框架。该两阶段方法明确针对延迟与模型大小,实现边缘设备高效部署。粗粒度阶段通过多目标深度剪枝移除整个注意力与MLP模块,降低计算负载与内存占用;细粒度阶段采用并行贝叶斯优化(PBO)在延迟约束下搜索最优逐层剪枝比例,结合重要性策略对每层内待剪组件进行排序。实验表明,本方法在最小化常识推理任务性能损失的同时,显著降低推理开销。在多个LLM上,于37.5%与50%剪枝率下,相比现有方法,在常识推理任务上表现更优,且大幅减少推理成本。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved widespread adoption because of their strong reasoning and query-response capabilities. However, deploying them in embedded and edge computing environments remains challenging because of strict latency, memory, and energy constraints. Their large parameter counts and computational demands hinder efficient execution on resource-constrained platforms. Although model pruning has emerged as a viable solution for reducing scale while preserving performance, jointly optimizing layers, attention heads, and Multi-Layer Perceptron (MLP) dimensions remains highly complex. Exhaustively exploring this combined design space is computationally expensive and often leads to local optima or unstable configurations. To address these limitations, we propose a hardware-aware, multi-objective structured pruning framework. The proposed two-stage method explicitly targets latency and model size for efficient deployment on edge devices. In the coarse-grained stage, multi-objective depth pruning removes entire attention and MLP blocks to reduce computational load and memory usage. In the subsequent fine-grained stage, Parallel Bayesian Optimization (PBO) searches for the optimal layer-wise pruning ratios for pruning under latency constraints, while importance-based strategies rank the specific components to be pruned within each layer's allocated budget. Experimental results show that our approach reduces model complexity with minimal impact on commonsense reasoning tasks and zero-shot performance. Our method achieves a favorable trade-off among accuracy, latency, and model size, making it suitable for edge deployment. Across multiple LLMs at 37.5% and 50% pruning ratios, the proposed approach achieves better performance on commonsense reasoning tasks than existing methods while significantly reducing inference cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。