用AI代理自动设计联邦学习系统,省去人工调参难题
Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agents
- 多智能体协作完成从需求到代码的全流程生成
- 在16项任务上生成方案优于或媲美人工设计基准
- 适合想快速搭建稳定联邦学习系统的研发团队
联邦学习(FL)可在分散数据上训练模型,但其系统设计与部署复杂度极高,常因数据异构性和系统约束导致方案脆弱且定制化严重。为解决此问题,我们提出Helmsman——一种多智能体系统,可从用户高层需求自动合成完整的联邦学习系统。该系统模拟科研开发流程,包含三个协同阶段:(1) 人机交互式规划制定研究方案;(2) 由监督智能体团队生成模块化代码;(3) 在沙盒环境中闭环评估并自主优化。为支持严谨评估,我们还构建了AgentFL-Bench,一个包含16个多样化任务的新基准,用于测试智能体系统在联邦学习中的系统级生成能力。大量实验表明,本方法生成的解决方案在性能上可媲美甚至超越已有手工设计基线。本工作标志着向自动化工程复杂分布式AI系统迈出关键一步。
原文摘要 · Abstract (English)
Federated Learning (FL) offers a powerful paradigm for training models on decentralized data, but its promise is often undermined by the immense complexity of designing and deploying robust systems. The need to select, combine, and tune strategies for multifaceted challenges like data heterogeneity and system constraints has become a critical bottleneck, resulting in brittle, bespoke solutions. To address this, we introduce Helmsman, a novel multi-agent system that automates the end-to-end synthesis of federated learning systems from high-level user specifications. It emulates a principled research and development workflow through three collaborative phases: (1) interactive human-in-the-loop planning to formulate a sound research plan, (2) modular code generation by supervised agent teams, and (3) a closed-loop of autonomous evaluation and refinement in a sandboxed simulation environment. To facilitate rigorous evaluation, we also introduce AgentFL-Bench, a new benchmark comprising 16 diverse tasks designed to assess the system-level generation capabilities of agentic systems in FL. Extensive experiments demonstrate that our approach generates solutions competitive with, and often superior to, established hand-crafted baselines. Our work represents a significant step towards the automated engineering of complex decentralized AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。