自进化平台自动构建并修复函数调用数据，提升大模型评估可信度。
EigenData: A Self-Evolving Multi-Agent Platform for Function-Calling Data Synthesis, Auditing, and Repair
- 多智能体架构协同生成真实数据库、可执行代码与多轮对话轨迹。
- 修复基准测试中3类系统性错误，使模型排名与人工判断更一致。
- 适合研究函数调用、评估可靠性及自动化数据治理的团队使用。
函数调用型智能体——即调用工具和API的大语言模型——需要高质量、领域特定的训练数据，覆盖可执行环境、后端数据库及多样化多轮交互轨迹。我们提出EigenData，一个集成的自进化平台，通过多智能体架构实现数据全生命周期自动化。顶层协调器EigenCore管理三个专用子系统：DatabaseAgent用于构建真实域数据库，CodingAgent通过迭代测试-调试循环生成经验证的可执行环境，DataAgent则结合自进化提示优化生成多轮轨迹。跨组件反馈确保所有成果一致性。我们将EigenData应用于审计和修复伯克利函数调用排行榜（BFCL-V3），识别出函数模式、实现和参考轨迹中的系统性错误，通过协调化的模式优化、代码级修复和轨迹调整自动修正，并引入结果感知评估协议，以数据库状态正确性而非逐轮轨迹匹配来评估任务成功。实验表明，修复后的基准与结果感知指标显著提升了模型排名与人工对功能正确性的判断的相关性。
原文摘要 · Abstract (English)
Function-calling agents -- large language models that invoke tools and APIs -- require high-quality, domain-specific training data spanning executable environments, backing databases, and diverse multi-turn trajectories. We introduce EigenData, an integrated, self-evolving platform that automates the full data lifecycle through a multi-agent architecture. A top-level orchestrator, EigenCore, coordinates three specialized sub-systems: DatabaseAgent for realistic domain database construction, CodingAgent for verified executable environment generation with iterative test-debug loops, and DataAgent for multi-turn trajectory synthesis with self-evolving prompt optimization. Cross-component feedback ensures consistency across all artifacts. We apply EigenData to audit and repair the Berkeley Function-Calling Leaderboard (BFCL-V3), identifying systematic errors in function schemas, implementations, and reference trajectories, automatically correcting them through coordinated schema refinement, code-level bug fixes, and trajectory modification, and introducing an outcome-aware evaluation protocol that assesses task success via database-state correctness rather than turn-level trajectory matching. We demonstrate that the repaired benchmark, coupled with outcome-aware metrics, produces model rankings substantially better correlated with human judgments of functional correctness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。