用隐空间连接扩散与自回归模型，提升多智能体推理效率
Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning
- 在隐空间中桥接扩散模型与自回归模型，实现规划与执行协同
- 在DART-5上准确率从27.0%提升至36.0%，AIME2024从0.0%到14.0%
- 仅用2.2%的词元预算，接近顶尖推理模型性能
多数多智能体系统依赖基于序列生成的自回归语言模型（ARMs），虽能生成流畅文本，但限制全局推理与计划修正。相反，离散扩散语言模型（DDLMs）支持非序列、可全局修订的生成，具备强规划能力，但文本流畅性不足，难以直接与ARMs协作。本文提出Latent-DARM，一种隐空间通信框架，连接DDLM（规划者）与ARM（执行者），最大化协同优势。在数学、科学及常识推理基准测试中，Latent-DARM平均优于基于文本接口的方法：DART-5准确率由27.0%提升至36.0%，AIME2024由0.0%升至14.0%。该方法接近当前最优推理模型表现，且词元预算低于其2.2%。本工作推动了异构模型间多智能体协作的发展。
原文摘要 · Abstract (English)
Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。