通过响应能量优化，实现状态空间模型的高效无训练剪枝。
AIRE-Prune: Asymptotic Impulse-Response Energy for State Pruning in State Space Models
- 基于无限时域响应能量评分，动态评估状态重要性。
- 平均剪枝率60.8%，精度仅降0.29%且无需重训练。
- 适合追求推理效率的SOTA状态空间模型部署者。
状态空间模型（SSMs）常因大状态维度带来高昂的内存与计算开销，需在容量、搜索空间或稳定性间做权衡。本文提出一种结构化后训练剪枝方法AIRE-Prune（渐近脉冲响应能量用于状态剪枝），通过直接最小化长期输出能量失真来降低每层状态维度。AIRE-Prune为每个状态分配闭式渐近脉冲响应能量得分，即其在无限时间内的总响应能量贡献，并进行层内归一化，以支持跨层全局比较与选择。该方法将模态截断从单系统扩展至深层堆叠结构，且剪枝标准由最坏情况增益转向渐近响应能量。在多种序列基准测试中，AIRE-Prune揭示了单输入单输出（SISO）与多输入多输出（MIMO） SSMs中存在显著冗余，平均剪枝率达60.8%，精度平均下降0.29%，且无需重训练，显著降低计算成本。代码已开源：https://github.com/falcon-arrow/AIRE-Prune。
原文摘要 · Abstract (English)
State space models (SSMs) often sacrifice capacity, search space, or stability to offset the memory and compute costs of large state dimensions. We introduce a structured post-training pruning method for SSMs -- AIRE-Prune (Asymptotic Impulse-Response Energy for State PRUN(E)) -- that reduces each layer's state dimension by directly minimizing long-run output-energy distortion. AIRE-Prune assigns every state a closed-form asymptotic impulse-response energy-based score, i.e., the total impulse-response energy it contributes over an infinite horizon (time), and normalizes these scores layer-wise to enable global cross-layer comparison and selection. This extends modal truncation from single systems to deep stacks and aligns pruning with asymptotic response energy rather than worst-case gain. Across diverse sequence benchmarks, AIRE-Prune reveals substantial redundancy in SISO and MIMO SSMs with average pruning of 60.8%, with average accuracy drop of 0.29% without retraining, while significantly lowering compute. Code: https://github.com/falcon-arrow/AIRE-Prune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。