让大模型学会预测动作后果，提升工具使用成功率与可靠性。
World Modelling Improves Language Model Agents
- 用内部环境模型预测动作未来状态，替代反复试错。
- 在伯克利工具调用榜单上，成功率显著提升，幻觉减少。
- 适合需要高可靠性的自动化任务与智能体开发人员。
在有状态环境中使用工具对大型语言模型（LLMs）带来独特挑战，现有依赖环境重复试错的推理时计算策略不切实际。我们提出动态建模（DyMo），在后训练阶段为LLM增加状态预测能力，结合函数调用。该方法使LLM能通过内部环境模型预测自身行为的未来状态。在伯克利函数调用排行榜V2上，DyMo显著提升了成功率并大幅减少幻觉。我们进一步将内部环境模型融入自验证采样（SVS），结果显示其显著提升pass^k指标随尝试次数k的增长效率，并支持模型拒绝不可靠输出。DyMo与SVS共同大幅提升LLM在工具使用中的有效性与可靠性。我们认为此工作为无需反复查询真实环境的可扩展规划强化学习方法铺平了道路。
原文摘要 · Abstract (English)
Tool use in stateful environments presents unique challenges for large language models (LLMs), where existing test-time compute strategies relying on repeated trials in the environment are impractical. We propose dynamics modelling (DyMo), a method that augments LLMs with a state prediction capability alongside function calling during post-training. This enables LLMs to predict the future states of their actions through an internal environment model. On the Berkeley Function Calling Leaderboard V2, DyMo improves success rates and significantly reduces hallucinations. We further integrate the internal environment model into self-verification sampling (SVS), and show that this substantially improves pass^k over number of trials k, and allows the model to refuse unreliable outputs. Together, DyMo and SVS greatly enhance the effectiveness and reliability of LLMs for tool use. We believe this work charts a path towards scalable planning RL methods for LLM inference without repeatedly querying the oracle environment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。