arXiv:2509.22631cs.CVcs.CL2025-09被引 6

用智能代理自动构建高质量计算机视觉数据集,省时省力。

LABELING COPILOT: A Deep Research Agent for Automated Data Curation in Computer Vision

  • 用大模型驱动的智能体分三步完成数据发现、合成与标注。
  • 在COCO上每图生成14.2个候选框,标注mAP达37.1%。
  • 适合需要大规模定制数据集的研究者和工业团队使用。

构建高质量、领域特定的数据集是部署鲁棒视觉系统的主要瓶颈,需在数据质量、多样性与成本间权衡。我们提出首个面向计算机视觉的数据集自动化整理智能体——Labeling Copilot。其核心是一个由大模态语言模型驱动的编排代理,通过多步推理调用三种核心能力:(1)校准发现,从大规模数据源中提取相关且分布一致的数据;(2)可控合成,针对罕见场景生成新数据并进行强过滤;(3)共识标注,通过融合多个基础模型的投票与非极大值抑制机制实现高精度标注。大规模验证表明其有效性:在密集的COCO数据集上,共识标注模块平均每张图生成14.2个候选框(接近双倍于7.4个真实物体),最终标注mAP达37.1%;在开放规模的Open Images数据集上,克服极端类别不平衡,成功发现903个新边界框类别,使总能力扩展至1500+类。同时,校准发现工具在千万级样本规模下测试,采用主动学习策略,计算效率较同类方法最高提升40倍,且样本效率相当。实验验证了优化、可扩展工具构成的智能体工作流,为工业级数据集构建提供了稳健基础。

原文摘要 · Abstract (English)

Curating high-quality, domain-specific datasets is a major bottleneck for deploying robust vision systems, requiring complex trade-offs between data quality, diversity, and cost when researching vast, unlabeled data lakes. We introduce Labeling Copilot, the first data curation deep research agent for computer vision. A central orchestrator agent, powered by a large multimodal language model, uses multi-step reasoning to execute specialized tools across three core capabilities: (1) Calibrated Discovery sources relevant, in-distribution data from large repositories; (2) Controllable Synthesis generates novel data for rare scenarios with robust filtering; and (3) Consensus Annotation produces accurate labels by orchestrating multiple foundation models via a novel consensus mechanism incorporating non-maximum suppression and voting. Our large-scale validation proves the effectiveness of Labeling Copilot's components. The Consensus Annotation module excels at object discovery: on the dense COCO dataset, it averages 14.2 candidate proposals per image-nearly double the 7.4 ground-truth objects-achieving a final annotation mAP of 37.1%. On the web-scale Open Images dataset, it navigated extreme class imbalance to discover 903 new bounding box categories, expanding its capability to over 1500 total. Concurrently, our Calibrated Discovery tool, tested at a 10-million sample scale, features an active learning strategy that is up to 40x more computationally efficient than alternatives with equivalent sample efficiency. These experiments validate that an agentic workflow with optimized, scalable tools provides a robust foundation for curating industrial-scale datasets.

数据集构建智能体计算机视觉自动化标注

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。