无需重训，通过分类得分剪枝注意力键，加速3D检测模型推理。
Accelerate 3D Object Detection Models via Zero-Shot Attention Key Pruning
- 基于分类得分与注意力图乘积计算键重要性，逐层剪枝键向量。
- 在ToC3D模型上实现1.99倍加速，性能损失小于1%。
- 可直接部署于边缘设备，适合资源受限场景的3D检测应用。
基于查询的3D目标检测方法虽取得显著成效，但其高计算开销（尤其在大图像尺寸和多层Transformer下）限制了在边缘设备上的高效运行。现有剪枝与蒸馏方法或需重新训练，或仅适用于ViT模型，难以迁移至3D检测器。为此，本文提出一种零样本运行时剪枝方法tgGBC（trim keys gradually Guided By Classification scores），通过将分类得分扩展并乘以注意力图，获得每条键的重要性评分，并按此分数逐层剪枝。该方法在最新ToC3D模型的Transformer解码器中实现1.99倍加速，性能损失低于1%；部分模型甚至性能提升。此外，我们在边缘设备上成功部署了采用tgGBC的3D检测器，验证了其有效性。代码已开源：https://github.com/iseri27/tg_gbc。
原文摘要 · Abstract (English)
Query-based methods with dense features have demonstrated remarkable success in 3D object detection tasks. However, the computational demands of these models, particularly with large image sizes and multiple transformer layers, pose significant challenges for efficient running on edge devices. Existing pruning and distillation methods either need retraining or are designed for ViT models, which are hard to migrate to 3D detectors. To address this issue, we propose a zero-shot runtime pruning method for transformer decoders in 3D object detection models. The method, termed tgGBC (trim keys gradually Guided By Classification scores), systematically trims keys in transformer modules based on their importance. We expand the classification score to multiply it with the attention map to get the importance score of each key and then prune certain keys after each transformer layer according to their importance scores. Our method achieves a 1.99x speedup in the transformer decoder of the latest ToC3D model, with only a minimal performance loss of less than 1%. Interestingly, for certain models, our method even enhances their performance. Moreover, we deploy 3D detectors with tgGBC on an edge device, further validating the effectiveness of our method. The code can be found at https://github.com/iseri27/tg_gbc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。