通过精简Mamba模型组件，实现高效推理加速。
Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models
- 分析Mamba模型各组件冗余性，按粒度删减无用部分
- 推理速度最高提升1.4倍，精度损失极小
- 适合追求部署效率的工业级模型优化场景
大规模预训练模型在序列建模中取得了卓越成果，其核心依赖于Transformer及其注意力机制。近年来，选择性结构状态空间模型（SSMs）等替代架构被提出以解决Transformer的效率瓶颈。本文研究基于SSM的模型（特别是Mamba及其混合结构）的压缩方法，通过分析不同粒度下移除特定组件对模型性能的影响，降低模型规模与计算开销，从而提升效率并保持精度。提出的综合方案统称为Mamba-Shedder，可在推理阶段实现最高1.4倍的速度提升，证明通过消除若干冗余项可显著改善模型效率，且对整体性能影响甚微。代码已公开于https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning。
原文摘要 · Abstract (English)
Large pre-trained models have achieved outstanding results in sequence modeling. The Transformer block and its attention mechanism have been the main drivers of the success of these models. Recently, alternative architectures, such as Selective Structured State Space Models (SSMs), have been proposed to address the inefficiencies of Transformers. This paper explores the compression of SSM-based models, particularly Mamba and its hybrids. We study the sensitivity of these models to the removal of selected components at different granularities to reduce the model size and computational overhead, thus improving their efficiency while maintaining accuracy. The proposed solutions, collectively referred to as Mamba-Shedder, achieve a speedup of up to 1.4x during inference, demonstrating that model efficiency can be improved by eliminating several redundancies with minimal impact on the overall model performance. The code is available at https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。