arXiv:2606.01279cs.AI2026-06

让AI agent自动生成高质量对齐数据,提升大模型训练效果

ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment

论文配图:ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment
图 1 · 摘自论文原文
  • 将数据生成设计为可插拔的agent技能,降低复杂性
  • 在有限算力下使弱基础agent达到顶尖对齐性能
  • 适合研究自主智能体与大模型对齐的开发者

AI代理正被用于自动化人工智能研究,尤其是将基础大模型转化为对齐助手的关键后训练阶段。然而,最新评估显示,即使前沿代理也难以胜任此任务。后训练的成功依赖于高质量数据,但让代理从开放网络中自主构建目标训练数据集面临严峻挑战。在嘈杂网络环境中执行长周期的数据搜索、过滤和平衡任务,常超出代理有限上下文,导致数据质量下降和下游训练性能不佳。为此,我们提出Andes(Agent Native Data Evolving Synthesis),一个将数据生成重构为即插即用的智能代理技能的框架。通过自演化世界树路由机制与可操作诊断报告,提供交互式闭环接口,使训练代理能动态引导数据合成。我们在严格计算约束下验证:赋予基础较弱的代理Andes后,可实现自动化对齐,并在PostTrainBench上取得当前最佳表现,具备强跨任务泛化能力。

原文摘要 · Abstract (English)

AI agents are increasingly being tasked with automating AI research itself, particularly the critical post-training phase that transforms base LLMs into aligned assistants. However, recent evaluations reveal that even frontier agents struggle to perform this task. While the success of post-training fundamentally relies on acquiring high-quality data, relying on agents to autonomously curate targeted training datasets from the open web introduces severe challenges. Executing the long-horizon tasks of searching, filtering, and balancing data within noisy web environments frequently overwhelms an agent's limited context, ultimately leading to degraded dataset quality and suboptimal downstream training performance. To bridge this gap, we introduce Andes (Agent Native Data Evolving Synthesis), a framework that reimagines data generation as a plug-and-play \emph{agent skill}. Rather than forcing agents to devise complex data-gathering strategies from scratch, \textsc{Andes} provides an intelligent abstraction layer. By leveraging a self-evolving World Tree routing mechanism and actionable diagnostic reports, it allows trainer agents to dynamically steer data synthesis through an interactive, closed-loop interface. We demonstrate that under strict compute constraints, equipping foundationally weaker agents with Andes improves automated alignment, securing state-of-the-art performance on PostTrainBench and robust cross-task generalization. Our project is available at https://github.com/zzy1127/ANDES.

AI代理数据合成模型对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。