arXiv:2508.04424cs.CV2025-08

提出细粒度物体级检索任务,实现像素级精准定位。

Composed Object Retrieval: Object-level Retrieval via Composed Expressions

  • 基于参考物体与文本描述进行组合视觉推理
  • 在12.5万条数据上达到领先性能,新基准支持类别泛化评估
  • 适合需要精准对象定位的多模态应用开发者

基于用户意图检索细粒度视觉内容仍是多模态系统中的挑战。现有组合图像检索(CIR)方法虽结合参考图像与检索文本,但仅限图像级匹配,无法定位具体物体。为此,我们提出组合物体检索(COR),一项新的物体级检索任务,旨在从目标图像的候选物体中检索目标物体,并以像素级掩码标注结果。给定参考物体、其掩码、目标图像及描述期望修改的检索文本,COR要求模型进行组合式视觉-文本推理,而非依赖显式类别名称。该设定引入多个挑战:细粒度组合匹配、在视觉相似干扰物下的负向物体过滤、以及单/多物体灵活检索。我们构建了首个大规模COR基准COR125K,包含跨408类的125,541个检索三元组,并提供基础/新类别划分以评估类别泛化能力。我们还提出了CORE,一种统一端到端模型,集成参考区域编码、自适应视觉-文本交互和区域级对比学习,以对齐组合表示并抑制背景与干扰物。大量实验表明,CORE在基础与新类别上均显著优于现有CIR管道与强基线,为细粒度物体级多模态检索建立了简单有效的基础。代码将公开于https://github.com/wangtong627/COR。

原文摘要 · Abstract (English)

Retrieving fine-grained visual content based on user intent remains a challenge in multimodal systems. Although current Composed Image Retrieval (CIR) methods combine reference images with retrieval texts, they are constrained to image-level matching and cannot localize specific objects. To this end, we propose Composed Object Retrieval (COR), a new object-level retrieval task that retrieves target object(s) from candidate objects in a target image and grounds the retrieved result with pixel-level masks. Given a reference object, its mask, a target image, and a retrieval text describing the desired modification, COR requires models to perform composed visual-textual reasoning rather than relying on explicit category names. This setting introduces several challenges, including fine-grained compositional matching, negative-object filtering under visually similar distractors, and flexible single- or multi-object retrieval. We construct COR125K, the first large-scale COR benchmark, containing 125,541 retrieval triplets across 408 categories with base/novel splits for evaluating category-level generalization. We also present CORE, a unified end-to-end model that integrates reference region encoding, adaptive vision-text interaction, and region-level contrastive learning to align composed representations with target objects while suppressing background and distractors. Extensive experiments demonstrate that CORE significantly outperforms existing CIR-based pipelines and strong baselines in both base and novel categories, establishing a simple and effective foundation for fine-grained object-level multimodal retrieval. Code will be released publicly at https://github.com/wangtong627/COR.

物体检索多模态细粒度像素定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。