HDINO无需人工标注数据,用简单方法实现高效开放词汇检测
HDINO: A Concise and Efficient Open-Vocabulary Detector
- 两阶段训练:先用噪声样本构建视觉文本对齐,再轻量融合特征增强语义敏感度
- 在COCO上达49.2 mAP,仅用220万图像,优于需540万、650万图像的方法
- 适合追求高效、低资源开放词汇检测的研究者和应用开发者
尽管近年来开放词汇目标检测备受关注,现有方法仍严重依赖人工标注的细粒度数据集及高耗能的逐层跨模态特征提取。本文提出HDINO,一种简洁高效的开放词汇检测器,完全摆脱这些依赖。具体而言,基于Transformer的DINO模型设计两阶段训练策略:第一阶段将噪声样本作为额外正样本,构建视觉与文本模态间的“一多语义对齐”机制,促进语义对齐;同时设计基于初始检测难度的困难样本加权分类损失(DWCL),挖掘硬样本以提升性能。第二阶段在对齐表示上引入轻量级特征融合模块,增强对语言语义的敏感性。在Swin Transformer-T设置下,HDINO-T仅使用来自两个公开检测数据集的220万张图像,在COCO上达到49.2 mAP,超越Grounding DINO-T(540万图像)和T-Rex2(650万图像)分别0.8和2.8 mAP。微调后,HDINO-T和HDINO-L分别达56.4 mAP和59.2 mAP,验证了方法的有效性与可扩展性。代码与模型见https://github.com/HaoZ416/HDINO。
原文摘要 · Abstract (English)
Despite the growing interest in open-vocabulary object detection in recent years, most existing methods rely heavily on manually curated fine-grained training datasets as well as resource-intensive layer-wise cross-modal feature extraction. In this paper, we propose HDINO, a concise yet efficient open-vocabulary object detector that eliminates the dependence on these components. Specifically, we propose a two-stage training strategy built upon the transformer-based DINO model. In the first stage, noisy samples are treated as additional positive object instances to construct a One-to-Many Semantic Alignment Mechanism(O2M) between the visual and textual modalities, thereby facilitating semantic alignment. A Difficulty Weighted Classification Loss (DWCL) is also designed based on initial detection difficulty to mine hard examples and further improve model performance. In the second stage, a lightweight feature fusion module is applied to the aligned representations to enhance sensitivity to linguistic semantics. Under the Swin Transformer-T setting, HDINO-T achieves \textbf{49.2} mAP on COCO using 2.2M training images from two publicly available detection datasets, without any manual data curation and the use of grounding data, surpassing Grounding DINO-T and T-Rex2 by \textbf{0.8} mAP and \textbf{2.8} mAP, respectively, which are trained on 5.4M and 6.5M images. After fine-tuning on COCO, HDINO-T and HDINO-L further achieve \textbf{56.4} mAP and \textbf{59.2} mAP, highlighting the effectiveness and scalability of our approach. Code and models are available at https://github.com/HaoZ416/HDINO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。