arXiv:2409.08475cs.CV2024-09被引 144

RT-DETRv3通过密集正样本监督提升实时目标检测精度,不增加推理延迟。

RT-DETRv3: Real-time End-to-End Object Detection with Hierarchical Dense Positive Supervision

  • 引入基于CNN的辅助分支,提供密集监督增强特征表示
  • 采用自注意力扰动策略,丰富查询组的标签分配,提升训练效果
  • 所有模块仅用于训练,实现在相同延迟下超越YOLOv10-X等模型

RT-DETR是首个实时端到端的Transformer目标检测器,其高效性源于框架设计与匈牙利匹配。然而,相比密集监督检测器(如YOLO系列),匈牙利匹配提供的监督稀疏,导致训练不足,难以达到最优性能。为此,我们提出基于RT-DETR的分层密集正样本监督方法RT-DETRv3:首先引入基于CNN的辅助分支,协同解码器增强编码器特征表示;其次提出一种自注意力扰动学习策略,通过多查询组间多样化标签分配,丰富正样本监督;此外,引入共享权重解码分支实现密集正样本监督,确保每个真实框有更多高质量查询匹配。上述模块均为训练阶段专用。在COCO val2017上大量实验表明,RT-DETRv3显著优于现有实时检测器。例如,RT-DETRv3-R18达到48.1% AP,较RT-DETR-R18/RT-DETRv2-R18分别提升1.6%/1.4%,且保持相同延迟;而RT-DETRv3-R101可达54.6% AP,超越YOLOv10-X。代码将开源于https://github.com/clxia12/RT-DETRv3。

原文摘要 · Abstract (English)

RT-DETR is the first real-time end-to-end transformer-based object detector. Its efficiency comes from the framework design and the Hungarian matching. However, compared to dense supervision detectors like the YOLO series, the Hungarian matching provides much sparser supervision, leading to insufficient model training and difficult to achieve optimal results. To address these issues, we proposed a hierarchical dense positive supervision method based on RT-DETR, named RT-DETRv3. Firstly, we introduce a CNN-based auxiliary branch that provides dense supervision that collaborates with the original decoder to enhance the encoder feature representation. Secondly, to address insufficient decoder training, we propose a novel learning strategy involving self-attention perturbation. This strategy diversifies label assignment for positive samples across multiple query groups, thereby enriching positive supervisions. Additionally, we introduce a shared-weight decoder branch for dense positive supervision to ensure more high-quality queries matching each ground truth. Notably, all aforementioned modules are training-only. We conduct extensive experiments to demonstrate the effectiveness of our approach on COCO val2017. RT-DETRv3 significantly outperforms existing real-time detectors, including the RT-DETR series and the YOLO series. For example, RT-DETRv3-R18 achieves 48.1% AP (+1.6%/+1.4%) compared to RT-DETR-R18/RT-DETRv2-R18, while maintaining the same latency. Furthermore, RT-DETRv3-R101 can attain an impressive 54.6% AP outperforming YOLOv10-X. The code will be released at https://github.com/clxia12/RT-DETRv3.

目标检测实时系统视觉Transformer密集监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。