arXiv:2609.01861cs.AI2026-09

让AI Agent显式记录对环境的判断,提升优化效率。

Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization

论文配图:Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization
图 1 · 摘自论文原文
  • 将Agent对环境反馈的信念写成可持久更新的文档,形成显式世界模型。
  • 在5个基准上比无世界模型的对照组提升训练通过率,且在所有预留数据集上保持优势。
  • 世界模型内容本身携带可复用信息,非仅形式作用,适合构建高效智能体系统。

LLM Agent的性能依赖于其外围框架。常见做法是使用编码代理作为优化器:读取当前评分与追踪信息,迭代修改源代码,每轮生成新候选。每次修改基于对环境响应的信念——哪里出错、如何改进。该信念通常隐含于推理过程或参数中,未被显式记录。后续调用虽见评分与追踪,却无法利用此前信念。本文提出信念校准优化(BCO),将信念以持续更新的上下文文档形式显式存储,形成对环境响应的动态建模。加入标准优化循环后,BCO在涵盖记忆问答、工具使用问答、代码即动作应用代理和终端代理的五个基准上,显著优于仅缺世界模型的对照组,且在所有预留数据划分上均保持差距。更换目标模型后,保留原架构的BCO仍主导测试任务,除非因上下文窗口溢出导致未完成。离线消融实验表明,该差距源于文档内容而非形式:基于累积文档的预测器,对环境响应的预测准确度高于无文档或内容被篡改的同形副本。结果说明文档内容本身蕴含可复用信息。

原文摘要 · Abstract (English)

The performance of an LLM agent depends on the scaffold around a frozen model. A common way to improve that scaffold is to use a coding agent as an optimizer: it reads current scores and traces and iteratively edits the source, producing a new candidate each round. Each edit is chosen according to a belief about how the environment will respond: what went wrong, and which change should help. That belief is typically implicit. It lives in the coding agent's reasoning on the current call, or remains latent in its parameters, rather than as something written down. Later calls therefore see scores and traces, but they do not use that belief. We introduce Belief-Calibrated Optimization (BCO), a method that writes that belief down as a persistent in-context document and continually revises that document as new candidates are evaluated. The resulting document is a world model: the current account of how the environment responds to edits. Added to an otherwise standard loop, BCO reaches a higher train passrate than a matched control that lacks only the world model, on five benchmarks spanning memory QA, tool-use QA, code-as-action app agents, and terminal agents. The gap remains on every held-out split, which is not used to select the candidate. After a target-model swap, in which the frozen model is replaced and the scaffold is not, the selected BCO scaffold leads on the tasks we test, except where context-window overruns leave it unfinished. An offline ablation then asks whether that gap comes from what the world model says. A fresh predictor given the accumulated document forecasts how the environment will respond more accurately than predictors given either no document or a same-form copy whose content has been falsified. The comparison indicates that the document carries reusable information in its content, not only in its form.

智能体世界模型优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。