针对推理阶段分拆特性，提出精准剪枝方法提升效率。
PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
- 基于预填充与解码分离特性设计迭代剪枝策略
- 在相同条件下实现更快推理与更强性能
- 适合部署于预填充-解码分离的LLM系统
大规模语言模型在各类任务中表现卓越，但其部署受制于高昂的计算与内存开销。模型剪枝是缓解此问题的有效手段。然而，现有方法常忽视实际中预填充-解码（PD）分离的特点。本文提出一种深度融入PD分离特性的剪枝方法，实现更精确的模块级剪枝。通过构建剪枝与蒸馏集，进行迭代块移除，获得更优剪枝方案。我们分析了预填充与解码阶段的剪枝敏感性，识别出各阶段可移除的特定模块，使该方法特别适用于PD分离部署。大量实验表明，本方法在PD分离与统一（非分离）两种设置下均保持优异性能，且可扩展至其他非块级剪枝方法。在相同配置下，实现更优性能与更快推理速度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning provides an effective means to alleviate these demands. However, existing methods often ignore the characteristics of prefill-decode (PD) disaggregation in practice. In this paper, we propose a pruning method that is highly integrated with PD disaggregation, enabling more precise pruning of blocks. Our approach constructs pruning and distillation sets to perform iterative block removal, obtaining better pruning solutions. Moreover, we analyze the pruning sensitivity of the prefill and decode stages and identify removable blocks specific to each stage, making it well suited for PD disaggregation deployment. Extensive experiments demonstrate our approach consistently achieves strong performance in both PD disaggregation and PD unified (non-PD disaggregation) settings, and can also be extended to other non-block pruning methods. Under the same settings, our method achieves improved performance and faster inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。