提出可学习的分块混合稀疏方法，让大模型在保持高精度的同时实现更快推理。
PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
- 将权重矩阵分块，用可学习掩码决定每块是否密集或2:4稀疏
- 在0.5B到13B模型上，比现有方法精度提升0.37%-2.96%，加速1.18倍至1.38倍
- 适合追求高效率与高精度平衡的LLM部署场景
大型语言模型（LLMs）性能卓越，但部署时带来高昂的内存和计算开销。模型剪枝是降低这些开销的有效手段，但现有方法面临挑战：非结构化稀疏性虽能保持精度，却导致访问模式不规则，无法有效利用GPU加速；半结构化2:4稀疏性硬件友好，但强制50%的固定模式会损害模型质量。为此，我们提出PATCH，一种混合稀疏框架，可在0%至50%之间实现连续稀疏率。PATCH将权重矩阵划分为多个块，通过可学习掩码机制决定每个块为密集或2:4稀疏。该设计实现了对精度-加速权衡的细粒度控制，并支持层间非均匀稀疏，从而获得更优的整体性能。在0.5B至13B参数模型上，PATCH持续缩小与密集模型的精度差距，同时实现实际加速。例如，在使用A6000 GPU的LLaMA-2 7B上，相比密集基线，PATCH实现1.18x-1.38x的端到端加速，且精度优于当前最优的2:4剪枝方法MaskLLM，提升0.37%-2.96%。
原文摘要 · Abstract (English)
Large language models (LLMs) deliver impressive performance but incur prohibitive memory and compute costs at deployment. Model pruning is an effective way to reduce these overheads, yet existing approaches face challenges: unstructured sparsity, where nonzeros can appear anywhere, preserves accuracy but yields irregular access patterns that prevent GPU acceleration, while semi-structured 2:4 sparsity is hardware-friendly but enforces a rigid 50% pattern that degrades model quality. To bridge this gap, we introduce PATCH, a hybrid sparsity framework that enables a continuous sparsity ratio between 0% and 50%. PATCH partitions weight matrices into tiles, assigning each tile to be either dense or 2:4 sparse via a learnable mask selection mechanism. This design provides fine-grained control over accuracy-acceleration tradeoffs and supports non-uniform sparsity across layers, leading to superior overall quality. Across models from 0.5B to 13B parameters, PATCH consistently narrows the gap to dense accuracy while delivering practical speedups. For instance, on LLaMA-2 7B with an A6000 GPU, PATCH achieves 1.18x-1.38x end-to-end speedup over dense baselines while improving accuracy by 0.37%-2.96% compared to the state-of-the-art 2:4 pruning method, MaskLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。