arXiv:2602.03137cs.CV2026-02中稿 · ICLR被引 2

用视觉大模型+图扩散优化少样本目标检测,提升准确率

FSOD-VFM: Few-Shot Object Detection with Vision Foundation Models and Graph Diffusion

  • 用通用框生成器+SAM2+DINOv2构建少样本检测框架
  • 10次拍摄下在CD-FSOD上达31.6 AP,远超之前21.4 AP
  • 图扩散重加权减少误检,无需额外训练

本文提出FSOD-VFM:一种利用视觉基础模型解决少样本目标检测的方法。该框架集成三个核心组件:通用提议网络(UPN)生成类别无关的边界框,SAM2实现精确掩码提取,DINOv2特征支持新类别的高效适应。尽管基础模型具备强泛化能力,但UPN生成的边界框常出现过度碎片化,仅覆盖物体局部区域,导致大量小而错误的提议框。为此,我们提出一种基于图的置信度重加权方法:将预测边界框建模为有向图节点,通过图扩散传播置信度分数,从而提升完整物体的置信度,降低局部碎片部分的置信度。该策略改善了检测粒度,有效减少误检框。在Pascal-5$^i$、COCO-20$^i$和CD-FSOD数据集上的大量实验表明,本方法显著优于现有方法,在不需额外训练的情况下性能领先。尤其在跨数据集与跨域的挑战性CD-FSOD数据集上,10次拍摄设置下达到31.6 AP,远超此前无训练方法的21.4 AP。代码已公开。

原文摘要 · Abstract (English)

In this paper, we present FSOD-VFM: Few-Shot Object Detectors with Vision Foundation Models, a framework that leverages vision foundation models to tackle the challenge of few-shot object detection. FSOD-VFM integrates three key components: a universal proposal network (UPN) for category-agnostic bounding box generation, SAM2 for accurate mask extraction, and DINOv2 features for efficient adaptation to new object categories. Despite the strong generalization capabilities of foundation models, the bounding boxes generated by UPN often suffer from overfragmentation, covering only partial object regions and leading to numerous small, false-positive proposals rather than accurate, complete object detections. To address this issue, we introduce a novel graph-based confidence reweighting method. In our approach, predicted bounding boxes are modeled as nodes in a directed graph, with graph diffusion operations applied to propagate confidence scores across the network. This reweighting process refines the scores of proposals, assigning higher confidence to whole objects and lower confidence to local, fragmented parts. This strategy improves detection granularity and effectively reduces the occurrence of false-positive bounding box proposals. Through extensive experiments on Pascal-5$^i$, COCO-20$^i$, and CD-FSOD datasets, we demonstrate that our method substantially outperforms existing approaches, achieving superior performance without requiring additional training. Notably, on the challenging CD-FSOD dataset, which spans multiple datasets and domains, our FSOD-VFM achieves 31.6 AP in the 10-shot setting, substantially outperforming previous training-free methods that reach only 21.4 AP. Code is available at: https://intellindust-ai-lab.github.io/projects/FSOD-VFM.

少样本检测视觉大模型图扩散目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。