arXiv:2607.21985cs.DCcs.AI2026-07

统一静态动态剪枝,让大模型推理更快更省电。

Unified Static-Dynamic Pruning for Efficient LLM Inference

论文配图:Unified Static-Dynamic Pruning for Efficient LLM Inference
图 1 · 摘自论文原文
  • 融合静态与动态剪枝,用新内存格式和双核优化推理
  • 比现有方法快1.24到2.51倍,支持高达25%的剪枝率
  • 适合追求高吞吐和低功耗的大模型部署场景

大语言模型(LLM)部署日益增加计算与内存瓶颈,自回归解码中低计算强度和带宽受限的内核主导了推理成本。权重剪枝提供了有效解决方案,但现有方法局限于静态剪枝(SP)——永久移除冗余权重但缺乏适应性,或动态剪枝(DP)——可适应输入稀疏性但引入运行时不规则性。本文提出SPDP,一种统一的稀疏推理框架,将无结构静态剪枝与输入自适应动态剪枝结合,用于GPU上的高效LLM推理。SPDP协同设计了一种新型分块列式位图压缩(Tiled-CBC)格式及两种互补的GPU内核:(1) 基于CUDA核心的spMspV内核,采用混合激活感知动态共享内存位图解码(HAD-SMBD),实现细粒度运行时激活跳过;(2) 针对预填充阶段优化的张量核心SpMM内核。该联合格式-内核设计协调了静态与动态稀疏性,在推理各阶段均保持高效的内存访问与高计算强度。在优化推理的GPU上全面评估显示,SPDP相比最先进稀疏框架SpInfer平均提速1.24至1.37倍(最高达2.51倍),在匹配困惑度的同时支持高达25%的剪枝率。SPDP推动了推理效率与质量的帕累托前沿,表明统一的静态-动态剪枝可在大规模LLM服务中显著提升吞吐量与每瓦性能。

原文摘要 · Abstract (English)

The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low compute intensity and bandwidth-bound kernels dominate inference cost. Weight pruning offers a promising remedy, but existing methods remain confined to either static pruning (SP), which permanently removes redundant weights but lacks adaptivity, or dynamic pruning (DP), which adapts to input sparsity but introduces runtime irregularity. This paper presents SPDP, a unified sparse-inference framework that integrates unstructured SP with input-adaptive DP for efficient LLM inference on GPUs. SPDP co-designs a new Tiled-Column-wise Bitmap Compressed (Tiled-CBC) format and two complementary GPU kernels: (1) a CUDA-core spMspV kernel featuring Hybrid Activation-aware Dynamic Shared-Memory Bitmap Decoding (HAD-SMBD) for fine-grained, runtime activation skipping, and (2) a Tensor-Core SpMM kernel optimized for prefill computation. This joint format-kernel design harmonizes static and dynamic sparsity, maintaining bandwidth-efficient memory access and high compute intensity under both phases of LLM inference. Comprehensive evaluations on inference-optimized GPUs demonstrate that SPDP achieves 1.24x-1.37x average speedup (up to 2.51x) over state-of-the-art sparse frameworks such as SpInfer, while matching perplexity with up to 25% higher sparsity. SPDP advances the inference efficiency-quality Pareto frontier, showing that unified static-dynamic pruning can deliver substantial throughput and performance-per-watt improvements in large-scale LLM serving.

大模型推理剪枝稀疏计算GPU优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。