轻量高效检测框架Le-DETR，用极少预训练实现顶尖实时检测性能。
Le-DETR: Revisiting Real-Time Detection Transformer with Efficient Encoder Design
- 设计高效编码器，融合局部注意力与轻量卷积结构
- 仅用ImageNet1K+COCO2017训练，预训练数据量减少80%
- 在RTX4090上达到55.1 mAP，速度提升20%
实时目标检测对实际应用至关重要，需兼顾高精度与低延迟。尽管检测变压器（DETR）已显著提升性能，但现有实时DETR模型因骨干网络过度预训练而难以复现，限制了新骨干结构的探索。本文提出通过通用优良设计，实现高性能与低预训练成本的平衡。我们研究骨干结构后，提出多尺度高效NAT（EfficientNAT），融合现代轻量卷积与局部注意力机制；并重新设计混合编码器，显著提升性能与推理速度。基于此，我们构建了低代价高效检测变压器（Le-DETR），仅使用ImageNet1K和COCO2017训练数据，在真实检测任务中达到新最优（SOTA）。相比以往方法，预训练图像量减少约80%。实验表明，Le-DETR-M/L/X在COCO Val2017上分别取得52.9/54.3/55.1 mAP，推理时间分别为4.45/5.01/6.68毫秒（RTX4090）。其性能超越YOLOv12-L/X（+0.6/-0.1 mAP）且速度相当，并实现20%提速；相较DEIM-D-FINE，Le-DETR-M提升+0.2 mAP且略快，而Le-DETR-L则以仅0.4毫秒额外延迟实现+0.4 mAP提升。代码与权重将开源。
原文摘要 · Abstract (English)
Real-time object detection is crucial for real-world applications as it requires high accuracy with low latency. While Detection Transformers (DETR) have demonstrated significant performance improvements, current real-time DETR models are challenging to reproduce from scratch due to excessive pre-training overheads on the backbone, constraining research advancements by hindering the exploration of novel backbone architectures. In this paper, we want to show that by using general good design, it is possible to have \textbf{high performance} with \textbf{low pre-training cost}. After a thorough study of the backbone architecture, we propose EfficientNAT at various scales, which incorporates modern efficient convolution and local attention mechanisms. Moreover, we re-design the hybrid encoder with local attention, significantly enhancing both performance and inference speed. Based on these advancements, we present Le-DETR (\textbf{L}ow-cost and \textbf{E}fficient \textbf{DE}tection \textbf{TR}ansformer), which achieves a new \textbf{SOTA} in real-time detection using only ImageNet1K and COCO2017 training datasets, saving about 80\% images in pre-training stage compared with previous methods. We demonstrate that with well-designed, real-time DETR models can achieve strong performance without the need for complex and computationally expensive pretraining. Extensive experiments show that Le-DETR-M/L/X achieves \textbf{52.9/54.3/55.1 mAP} on COCO Val2017 with \textbf{4.45/5.01/6.68 ms} on an RTX4090. It surpasses YOLOv12-L/X by \textbf{+0.6/-0.1 mAP} while achieving similar speed and \textbf{+20\%} speedup. Compared with DEIM-D-FINE, Le-DETR-M achieves \textbf{+0.2 mAP} with slightly faster inference, and surpasses DEIM-D-FINE-L by \textbf{+0.4 mAP} with only \textbf{0.4 ms} additional latency. Code and weights will be open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。