用大模型自动发现并标注未知物体，无需训练。
ADAM: Autonomous Discovery and Annotation Model using LLMs for Context-Aware Annotations
- 利用大模型结合上下文生成未知物体候选标签。
- 通过视觉相似性检索与投票机制实现精准标注。
- 自迭代优化标签一致性，适合开放世界场景。
目标检测模型通常依赖预定义类别，难以在开放世界中识别新物体。为此，我们提出ADAM：一种无需训练、自我优化的开放世界物体标注框架。ADAM利用大语言模型（LLMs）基于场景中已知实体的上下文信息，生成未知物体的候选标签，并将其与CLIP的视觉嵌入结合，构建嵌入-标签库（ELR），实现无类别监督推理。对于新出现的未知物体，ADAM从ELR中检索视觉相似实例，通过频率投票与跨模态重排序分配稳健标签。为进一步提升一致性，引入自优化循环，基于视觉连贯性分析和基于k近邻的多数重标注重新评估库内标签。在COCO和PASCAL数据集上的实验表明，ADAM仅依靠视觉与上下文信号即可有效标注新类别，无需微调或重训练。
原文摘要 · Abstract (English)
Object detection models typically rely on predefined categories, limiting their ability to identify novel objects in open-world scenarios. To overcome this constraint, we introduce ADAM: Autonomous Discovery and Annotation Model, a training-free, self-refining framework for open-world object labeling. ADAM leverages large language models (LLMs) to generate candidate labels for unknown objects based on contextual information from known entities within a scene. These labels are paired with visual embeddings from CLIP to construct an Embedding-Label Repository (ELR) that enables inference without category supervision. For a newly encountered unknown object, ADAM retrieves visually similar instances from the ELR and applies frequency-based voting and cross-modal re-ranking to assign a robust label. To further enhance consistency, we introduce a self-refinement loop that re-evaluates repository labels using visual cohesion analysis and k-nearest-neighbor-based majority re-labeling. Experimental results on the COCO and PASCAL datasets demonstrate that ADAM effectively annotates novel categories using only visual and contextual signals, without requiring any fine-tuning or retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。