通过快速重训练，几分钟内实现视觉Mamba模型高效压缩与性能恢复。
Faster Vision Mamba is Rebuilt in Minutes via Merged Token Re-training
- 合并令牌后快速重训练，避免信息丢失
- 模型压缩后准确率仅下降0.9%，3轮训练提升35.9%性能
- 小模型可在5分钟内完成重训练，推理速度最高提升1.5倍
视觉Mamba在计算机视觉任务中已接近顶尖性能，引发对其效率提升的关注。现有方法如令牌剪枝会损失关键知识，导致性能下降；而令牌合并虽能保留更多信息，但在高压缩比下仍表现不佳。本文提出R-MeeTo框架，通过合并令牌后快速重训练，在多种压缩比下均取得稳健效果。实验表明，剪枝后的Vim模型在ImageNet-1K上准确率最多仅下降0.9%，经本框架恢复后，仅用3个训练周期即实现35.9%的性能提升。Vim-Ti/S/B分别在5/7/17分钟内完成重训练，其中Vim-S在压缩后仅损失1.3%准确率，推理速度提升1.2倍（最高达1.5倍）。
原文摘要 · Abstract (English)
Vision Mamba has shown close to state of the art performance on computer vision tasks, drawing much interest in increasing it's efficiency. A promising approach is token reduction (that has been successfully implemented in ViTs). Pruning informative tokens in Mamba leads to a high loss of key knowledge and degraded performance. An alternative, of merging tokens preserves more information than pruning, also suffers for large compression ratios. Our key insight is that a quick round of retraining after token merging yeilds robust results across various compression ratios. Empirically, pruned Vims only drop up to 0.9% accuracy on ImageNet-1K, recovered by our proposed framework R-MeeTo in our main evaluation. We show how simple and effective the fast recovery can be achieved at minute-level, in particular, a 35.9% accuracy spike over 3 epochs of training on Vim-Ti. Moreover, Vim-Ti/S/B are re-trained within 5/7/17 minutes, and Vim-S only drops 1.3% with 1.2x (up to 1.5x) speed up in inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。