DEIM加速实时目标检测,训练一天达53.2%精度。
DEIM: DETR with Improved Matching for Fast Convergence

- 用密集一对一匹配增加正样本,提升收敛速度。
- 结合自适应损失函数,有效过滤低质量匹配。
- 无需额外数据,实现实时检测新标杆。
本文提出DEIM,一种针对基于Transformer的实时目标检测模型(如DETR)的高效训练框架,旨在加速收敛。为缓解DETR中一对一匹配带来的稀疏监督问题,DEIM采用密集一对一匹配策略,通过标准数据增强引入更多目标,增加每张图像的正样本数量。然而,该策略会带来大量低质量匹配。为此,我们设计了可匹配性感知损失(MAL),优化不同质量匹配的权重,提升密集匹配的有效性。在COCO数据集上的大量实验表明,将DEIM集成到RT-DETR和D-FINE中,可使训练时间减少50%,同时持续提升性能。特别地,与RT-DETRv2结合,在单块NVIDIA 4090 GPU上仅用一天训练即达53.2% AP。此外,DEIM训练的模型在NVIDIA T4 GPU上分别以124和78 FPS实现54.7%和56.5% AP,优于现有实时检测器,且无需额外数据。代码与预训练模型已开源。
原文摘要 · Abstract (English)
We introduce DEIM, an innovative and efficient training framework designed to accelerate convergence in real-time object detection with Transformer-based architectures (DETR). To mitigate the sparse supervision inherent in one-to-one (O2O) matching in DETR models, DEIM employs a Dense O2O matching strategy. This approach increases the number of positive samples per image by incorporating additional targets, using standard data augmentation techniques. While Dense O2O matching speeds up convergence, it also introduces numerous low-quality matches that could affect performance. To address this, we propose the Matchability-Aware Loss (MAL), a novel loss function that optimizes matches across various quality levels, enhancing the effectiveness of Dense O2O. Extensive experiments on the COCO dataset validate the efficacy of DEIM. When integrated with RT-DETR and D-FINE, it consistently boosts performance while reducing training time by 50%. Notably, paired with RT-DETRv2, DEIM achieves 53.2% AP in a single day of training on an NVIDIA 4090 GPU. Additionally, DEIM-trained real-time models outperform leading real-time object detectors, with DEIM-D-FINE-L and DEIM-D-FINE-X achieving 54.7% and 56.5% AP at 124 and 78 FPS on an NVIDIA T4 GPU, respectively, without the need for additional data. We believe DEIM sets a new baseline for advancements in real-time object detection. Our code and pre-trained models are available at https://github.com/ShihuaHuang95/DEIM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。