通过动态调整稀疏模式，提升大模型推理速度与精度。
Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator
- 根据层内异常值分布，自适应选择最优稀疏比例
- 实验显示准确率最高提升36%，延迟降低1.75倍
- 数字存算一体架构支持灵活稀疏配置，适合部署复杂模型
大型语言模型（LLM）采用固定N:M结构化稀疏化会严重限制模型表达能力，导致性能下降。而支持多种N:M模式虽能提升表达自由度，却带来高昂的硬件开销。为此，本文提出一种分层自适应的灵活稀疏选择方法（FLOW），通过考虑异常值的分布和存在性，为每一层动态选择最优的N和M值，从而增强模型表达能力。为实现此类灵活稀疏性，进一步设计了低开销的数字存算一体架构（FlexCiM），将数字存算宏（DCiM）划分为更小子宏，通过分配与合并机制按需重组，以适配不同稀疏模式。在基于Transformer和状态空间模型（SSMs）的广泛实验中，FLOW相比现有方法准确率最高提升36%；FlexCiM相较现有稀疏加速器，推理延迟降低1.75倍，能耗减少1.5倍。代码已开源：https://github.com/FLOW-open-project/FLOW
原文摘要 · Abstract (English)
Large language model (LLM) pruning with fixed N:M structured sparsity significantly limits the expressivity of the sparse model, yielding sub-optimal performance. In contrast, supporting multiple N:M patterns to provide sparse representational freedom introduces costly overhead in hardware. To address these challenges for LLMs, we first present a flexible layer-wise outlier-density-aware N:M sparsity (FLOW) selection method. FLOW enables the identification of optimal layer-wise N and M values (from a given range) by simultaneously accounting for the presence and distribution of outliers, allowing a higher degree of representational freedom. To deploy sparse models with such N:M flexibility, we then introduce a flexible, low-overhead digital compute-in-memory architecture (FlexCiM). FlexCiM supports diverse sparsity patterns by partitioning a digital CiM (DCiM) macro into smaller sub-macros, which are adaptively aggregated and disaggregated through distribution and merging mechanisms for different N and M values. Extensive experiments on both transformer-based and recurrence-based state space foundation models (SSMs) demonstrate that FLOW outperforms existing alternatives with an accuracy improvement of up to 36%, while FlexCiM achieves up to 1.75x lower inference latency and 1.5x lower energy consumption compared to existing sparse accelerators. Code is available at: https://github.com/FLOW-open-project/FLOW
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。