用预训练ViT提升人机协作找图效率,精准定位复杂场景中的目标物体。
Revisiting Human-in-the-Loop Object Retrieval with Pre-Trained Vision Transformers

- 基于预训练ViT提取局部特征,增强对小目标的识别能力。
- 通过主动学习选择关键样本,用户只需少量反馈即显著提升检索准确率。
- 适用于多物体复杂图像场景,适合需要高效标注的视觉检索应用。
本文重新审视人机协同的目标检索任务:从大规模无标签图像集合出发,仅凭初始查询和用户相关性反馈,快速识别某类物体的多样实例。该过程被建模为二分类任务,系统通过迭代用户交互持续学习区分相关与无关图像。交互由主动学习循环驱动,在每轮中系统选择信息量大的样本供用户标注,从而优化检索性能。在包含多个物体的复杂场景中,目标可能仅占图像局部区域,传统全局描述符效果有限,需依赖更精细的局部特征表示。本文利用预训练视觉变压器(ViT)重构该任务,探讨关键设计问题:应考虑图像中哪些目标实例、标注形式如何、主动选择策略怎样应用,以及何种表征能更好捕捉物体特征。在多物体数据集上对比多种表征策略,揭示全局上下文与细粒度局部细节之间的权衡。实验结果为基于主动学习的物体类别检索系统设计提供了实用指导。
原文摘要 · Abstract (English)
Building on existing approaches, we revisit Human-in-the-Loop Object Retrieval, a task that consists of iteratively retrieving images containing objects of a class-of-interest, specified by a user-provided query. Starting from a large unlabeled image collection, the aim is to rapidly identify diverse instances of an object category relying solely on the initial query and the user's Relevance Feedback, with no prior labels. The retrieval process is formulated as a binary classification task, where the system continuously learns to distinguish between relevant and non-relevant images to the query, through iterative user interaction. This interaction is guided by an Active Learning loop: at each iteration, the system selects informative samples for user annotation, thereby refining the retrieval performance. This task is particularly challenging in multi-object datasets, where the object of interest may occupy only a small region of the image within a complex, cluttered scene. Unlike object-centered settings where global descriptors often suffice, multi-object images require more adapted, localized descriptors. In this work, we formulate and revisit the Human-in-the-Loop Object Retrieval task by leveraging pre-trained ViT representations, and addressing key design questions, including which object instances to consider in an image, what form the annotations should take, how Active Selection should be applied, and which representation strategies best capture the object's features. We compare several representation strategies across multi-object datasets highlighting trade-offs between capturing the global context and focusing on fine-grained local object details. Our results offer practical insights for the design of effective interactive retrieval pipelines based on Active Learning for object class retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。