将多智能体推理能力提炼进单个大模型，实现高效且自纠错的智能。
AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent
- 通过三种分层蒸馏策略，把多智能体互动转化为模型权重中的隐式能力。
- 蒸馏后模型在多种任务上保持高效，同时具备多智能体的强推理与自我修正能力。
- 适合追求高效、鲁棒推理的AI系统开发者使用。
尽管大型语言模型多智能体系统通过迭代辩论实现优异的推理性能，但其高计算成本和错误传播限制了实际部署。本文提出AgentArk，一种新框架，将多智能体动态蒸馏至单个模型权重中，将显式的测试时交互转变为隐式的模型能力。这使单一智能体具备多智能体系统的智能，同时保持计算高效性。我们研究了三种跨不同模型、任务、规模和场景的分层蒸馏策略：增强推理的微调、基于轨迹的增强以及过程感知蒸馏。通过将计算负担从推理阶段转移到训练阶段，蒸馏后的模型在保持单智能体效率的同时，展现出多智能体的强推理与自我修正能力，并在多样推理任务中表现出更强的鲁棒性和泛化能力。我们希望本工作能为未来高效、鲁棒的多智能体开发提供启发。代码已开源：https://github.com/AIFrontierLab/AgentArk。
原文摘要 · Abstract (English)
While large language model (LLM) multi-agent systems achieve superior reasoning performance through iterative debate, practical deployment is limited by their high computational cost and error propagation. This paper proposes AgentArk, a novel framework to distill multi-agent dynamics into the weights of a single model, effectively transforming explicit test-time interactions into implicit model capabilities. This equips a single agent with the intelligence of multi-agent systems while remaining computationally efficient. Specifically, we investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios: reasoning-enhanced fine-tuning; trajectory-based augmentation; and process-aware distillation. By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents. They further demonstrate enhanced robustness and generalization across diverse reasoning tasks. We hope this work can shed light on future research on efficient and robust multi-agent development. Our code is at https://github.com/AIFrontierLab/AgentArk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。