解决多智能体大模型协作微调中的上下文漂移问题，提升协同性能。
TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination
- 引入信任域框架，每次更新后重采样轨迹并控制各智能体分布偏移
- 实验显示平均性能提升7.1%，且避免协作退化
- 适合需要模块化替换与稳定协作的多智能体系统研发
多智能体大模型系统在复杂推理中展现出潜力，但近期评估发现其常逊于单模型基线。我们识别出共享上下文团队顺序微调中的结构性缺陷：更新一个智能体会改变团队上下文分布，当后续更新在缓存回放上评估时，这种不匹配会累积。我们将其形式化为复合占据漂移，并证明静态占用评估的惩罚随智能体数量呈平方增长；而中间占用评估可将该增长降至线性。我们提出TeamTR，一种信任域框架，通过每次组件更新后重采样轨迹并施加每智能体差异约束，实现严格的每步与每阶段改进下界。实验表明，TeamTR平均性能优于单智能体与顺序基线7.1%，缓解了协作退化，并支持即插即用组件替换。代码已开源于https://github.com/Yydc/TeamTR。
原文摘要 · Abstract (English)
Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updating one agent shifts the team's context distribution, and when subsequent updates are evaluated on cached rollouts, this mismatch compounds. We formalize this as the compounding occupancy shift and prove that stale-occupancy evaluation incurs a penalty that scales quadratically with the number of agents. In contrast, intermediate-occupancy evaluation reduces this to linear scaling. We propose TeamTR, a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control, yielding rigorous per-update and per-stage improvement lower bounds. Experiments show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement. Code is available at https://github.com/Yydc/TeamTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。