arXiv:2606.04261cs.AIcs.CL2026-06被引 2

通用编码代理可自动完成数据清洗循环,但需结构化引导才能突破创新。

Can Generalist Agents Automate Data Curation?

论文配图:Can Generalist Agents Automate Data Curation?
图 1 · 摘自论文原文
  • 用指令调优的视觉语言模型实现自动数据筛选,十轮内达现有基线水平
  • 仅调整局部策略,难以探索新方法,存在执行与研究脱节问题
  • 引入方法引用和模板后,代理自主生成高效策略,数据量减少90%仍更优

数据清洗是现代AI开发中最关键也最耗人力的环节:从业者需反复提出、实施、评估并修正数据策略,依赖噪声较大的基准反馈。本文探讨通用编码代理是否能自动化这一流程。提出*Curation-Bench*——以代理为中心的基准测试,固定模型、训练方案与评估套件,仅开放命令行权限供代理检查数据、制定策略、提交至固定训练/评估流水线并迭代优化。在视觉-语言指令微调场景中,无需微调的代理在十轮内达到已有公开数据筛选基线性能。但轨迹分析显示存在持续的‘执行-研究差距’:即使提供策略指南与论文参考,代理仍主要在局部策略上微调,未探索新策略族。引入要求每轮引用、实例化并适配先前方法的支架后,代理转向方法导向探索,自主设计出超越强基线的策略,仅用十分之一的数据预算即达成更优效果。总体表明,当前代理可运行清洗循环,但可靠数据研究需依赖支架式方法迁移,而非开放式提示。代码与基准已开源。

原文摘要 · Abstract (English)

Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback. We ask whether generalist coding agents can automate this data-curation loop. We introduce *Curation-Bench*, an agent-centric benchmark that fixes the model, training recipe, and evaluation suite while giving agents command-line access to inspect data, implement policies, submit them to a fixed training/evaluation pipeline, and revise. In a vision-language instruction-tuning instantiation, out-of-the-box agents reach strong published data-selection baselines within ten iterations. However, trajectory analysis reveals a persistent *execution-research gap*: agents mainly tune local policy variants rather than explore new policy families, even when given strategy guides and paper references. Scaffolds requiring each iteration to cite, instantiate, and adapt a prior method shift agents toward method-guided exploration. The scaffolded agent autonomously composes -- without human design input -- a data-selection policy that outperforms strong published baselines at one-tenth their data budget. Overall, current agents can run the curation loop, but reliable data research requires scaffolded method adaptation, not open-ended prompting alone. Code and benchmark are open-sourced.

数据清洗智能代理自动化机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。