让预训练检测模型更高效：动态稀疏化不重训也能提速
Utilizing dynamic sparsity on pretrained DETR
- 用轻量门控机制预测输入相关的神经元激活状态
- 在COCO上实现85%-95%激活稀疏度，计算量大幅下降
- 无需重训练，适合部署已有视觉Transformer模型
基于Transformer的模型高效推理仍是挑战，尤其在目标检测等视觉任务中。本文分析了DETR中MLP层的固有稀疏性，提出两种无需重训练即可利用该稀疏性的方法。首先提出静态指示器稀疏化（SIBS），基于固定激活模式预测神经元不活跃状态，但受限于输入依赖性，增益有限。为此，提出微门控稀疏化（MGS），一种在预训练DETR之上训练的轻量门控机制，通过小型线性层预测动态稀疏性，在COCO数据集上的实验表明，MGS可实现85%至95%的激活稀疏度，同时保持甚至提升性能，显著降低计算开销。该方法提供了一种实用且输入自适应的稀疏化方案，支持预训练视觉Transformer的高效部署。
原文摘要 · Abstract (English)
Efficient inference with transformer-based models remains a challenge, especially in vision tasks like object detection. We analyze the inherent sparsity in the MLP layers of DETR and introduce two methods to exploit it without retraining. First, we propose Static Indicator-Based Sparsification (SIBS), a heuristic method that predicts neuron inactivity based on fixed activation patterns. While simple, SIBS offers limited gains due to the input-dependent nature of sparsity. To address this, we introduce Micro-Gated Sparsification (MGS), a lightweight gating mechanism trained on top of a pretrained DETR. MGS predicts dynamic sparsity using a small linear layer and achieves up to 85 to 95% activation sparsity. Experiments on the COCO dataset show that MGS maintains or even improves performance while significantly reducing computation. Our method offers a practical, input-adaptive approach to sparsification, enabling efficient deployment of pretrained vision transformers without full model retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。