arXiv:2608.30247cs.CV2026-08

用简单框架实现多模态提示的统一检测,效果超越现有方法。

OPUS: A Simple yet Effective Unified Framework for Open-Vocabulary Detection

论文配图:OPUS: A Simple yet Effective Unified Framework for Open-Vocabulary Detection
图 1 · 摘自论文原文
  • 采用语义丰富的视觉编码与统一提示解码器,避免复杂分支设计。
  • 在COCO等数据集上达到68.1/69.2/54.7 AP,文本与视觉提示均衡表现。
  • 混合提示由冲突转为互补,提升检测性能,适合开放词汇场景应用。

近期统一开放词汇检测(OVD)支持文本、视觉样例及其组合的异构提示,但常依赖日益复杂的结构,如重型跨模态融合、分阶段训练和迭代标注流程。我们重新审视在强基础模型时代,这种复杂性是否必要。结果表明,通过语义丰富的视觉表征和可扩展的定位监督,统一OVD可显著简化。我们提出OPUS(Open-vocabulary, Prompt-Unified, Simple),一个支持文本、交互式视觉、通用视觉及混合提示的统一检测框架。OPUS采用三部分简洁设计:基于DINOv3-ConvNeXt-B主干的语义丰富视觉编码器,结合高效混合编码;无需特定提示分支的提示感知解码器,实现统一提示推理。通过单阶段文本-视觉训练策略与实例级对比对齐(ICA)训练,并依托SAM3-based单次通过数据引擎获取异构定位监督。在COCO、LVIS-minival和ODinW35上的实验显示,OPUS在视觉-图像(Visual-I)任务上达到68.1/69.2/54.7 AP,同时保持文本与视觉-通用提示的平衡精度。此外,混合提示从干扰转为互补,优于单独使用文本或视觉提示。

原文摘要 · Abstract (English)

Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but often rely on increasingly complex designs such as heavy cross-modal fusion, staged training, and iterative annotation pipelines. We revisit whether such complexity is necessary in the era of stronger foundation models. Our finding is that unified OVD can be made substantially simpler with semantic-rich visual representations and scalable grounding supervision. We present OPUS (\textbf{O}pen-vocabulary, \textbf{P}rompt-\textbf{U}nified, \textbf{S}imple), a unified detector supporting text, interactive visual, generic visual, and mixed prompting within one framework. OPUS adopts a simple three-part design. Its model architecture combines a semantic-rich visual encoder, built on a DINOv3-ConvNeXt-B backbone with efficient hybrid encoding, with a prompt-aware decoder that avoids prompt-specific branches for unified prompt reasoning. OPUS is trained with a one-stage text-visual training strategy with Instance-level Contrastive Alignment (ICA), and is supported by a SAM3-based single-pass data engine for heterogeneous grounding supervision. Experiments on COCO, LVIS-minival, and ODinW35 show that OPUS achieves state-of-the-art Visual-I performance, reaching 68.1/69.2/54.7 AP, while maintaining balanced Text and Visual-G accuracy. OPUS also turns mixed prompting from interference into complementarity, improving over text or visual prompt alone. These results show that simplicity and strong unified prompting capability can be achieved together.

目标检测开放词汇多模态提示统一框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。