在隐私保护下实现多环境大模型智能体的协同进化。
Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents
- 本地筛选高回报轨迹，用高效微调稳定更新梯度。
- 全局聚合在低秩子空间进行，通信开销更低。
- 跨五种环境提升任务成功率10%，适合隐私敏感场景。
大模型智能体广泛应用于复杂交互任务，但隐私约束常阻碍集中式优化与跨动态环境协同演化。尽管联邦学习（FL）在静态数据集上表现良好，但在开放、自演化的智能体系统中仍缺乏探索。标准FL在此类场景面临严峻挑战：任务异构性与稀疏的轨迹级奖励信号导致梯度不稳定，破坏全局优化。为此，我们提出Fed-SE，一种面向大模型智能体的联邦自演化框架，建立本地演化-全局聚合范式。本地端，智能体对筛选出的高回报轨迹进行参数高效微调，实现稳定梯度更新；全局端，通过低秩子空间聚合更新，降低客户端间通信开销。在五个异构环境上的实验表明，相比最先进的FedIT，Fed-SE平均任务成功率提升10%，验证了其在隐私约束下的跨环境知识迁移有效性。
原文摘要 · Abstract (English)
LLM agents are widely deployed in complex interactive tasks, yet privacy constraints often preclude centralized optimization and co-evolution across dynamic environments. Despite the demonstrated success of Federated Learning (FL) on static datasets, its effectiveness in open-ended, self-evolving agent systems remains largely unexplored. In such settings, the direct application of standard FL is particularly challenging, as heterogeneous tasks and sparse, trajectory-level reward signals give rise to severe gradient instability, which undermines the global optimization process. To bridge this gap, we propose Fed-SE, a Federated Self-Evolution framework for LLM agents that establishes a local evolution-global aggregation paradigm. Locally, agents employ parameter-efficient fine-tuning on filtered, high-return trajectories to achieve stable gradient updates. Globally, Fed-SE aggregates updates within a low-rank subspace, reducing communication cost across clients. Experiments across five heterogeneous environments demonstrate that Fed-SE improves average task success rates by 10\% over the state-of-the-art FedIT, validating its effectiveness in cross-environment knowledge transfer under privacy constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。