GHOST通过前向统计实现结构化剪枝，大幅降低Mamba2推理开销。
GHOST: Unmasking Phantom States in Mamba2 via Grouped Hidden-state Output-aware Selection & Truncation
- 基于前向统计的结构化剪枝，无需反向传播
- 130M~2.7B参数模型下状态维度减半，困惑度仅增约1点
- 适合追求高效推理的Mamba2应用开发者
尽管Mamba2扩展的状态维度提升了时间建模能力，但其带来的巨大推理开销会饱和带宽，尤其在自回归生成时。标准剪枝方法难以解决此瓶颈：非结构化稀疏性使激活值仍密集，基于幅度的选择忽略运行时动态，梯度方法则成本过高。我们提出GHOST（分组隐藏状态输出感知选择与截断），一种仅使用前向传递统计信息的结构化剪枝框架，近似控制理论中的平衡截断。通过联合衡量可控性和可观测性，GHOST在不需反向传播的情况下，媲美梯度方法的精度。作为亮点，在130M至2.7B参数的模型上，该方法实现了50%的状态维度缩减，且在WikiText-2上的困惑度仅增加约1点。代码已公开于https://anonymous.4open.science/r/mamba2_ghost-7BCB/。
原文摘要 · Abstract (English)
While Mamba2's expanded state dimension enhances temporal modeling, it incurs substantial inference overhead that saturates bandwidth during autoregressive generation. Standard pruning methods fail to address this bottleneck: unstructured sparsity leaves activations dense, magnitude-based selection ignores runtime dynamics, and gradient-based methods impose prohibitive costs. We introduce GHOST (Grouped Hidden-state Output-aware Selection and Truncation), a structured pruning framework that approximates control-theoretic balanced truncation using only forward-pass statistics. By jointly measuring controllability and observability, GHOST rivals the fidelity of gradient-based methods without requiring backpropagation. As a highlight, on models ranging from 130M to 2.7B parameters, our approach achieves a 50\% state-dimension reduction with approximately 1 perplexity point increase on WikiText-2. Code is available at https://anonymous.4open.science/r/mamba2_ghost-7BCB/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。