让大模型自己决定要学什么，比预测下一个画面更有效。
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
- 模型主动识别决策前需要理解的环境动态
- 用决策所需信息构建训练目标，优于传统观测预测
- 适合需要精准决策的智能体系统，如机器人、游戏AI
当前大语言模型智能体的世界建模通常以预测下一个观测为学习目标，但该目标依赖于状态转移的显性呈现，可能忽略对当前决策至关重要的动态。为此，我们提出代理自著世界建模（AAWM），通过智能体自身决策需求生成监督信号。在每个状态，智能体识别行动前所需理解的环境信息，基于这些需求跨轨迹检索相关转移证据，并将其合成决策导向的训练目标，而非重建下一个观测。该方法使训练目标与智能体决策前所需动态对齐，而非与下一时刻观测内容对齐。实验结果表明，在多个环境和训练设置下，决策感知的世界模型目标均显著优于传统的下一步观测预测，验证了其有效性。
原文摘要 · Abstract (English)
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties supervision to what a transition happens to reveal, which may omit the dynamics most relevant to the agent's current decision. To bridge this gap, we propose Agent-Authored World Modeling (AAWM), a training procedure that constructs supervision from the policy's own decision needs. Specifically, at each state, the agent identifies what it needs to understand about the environment before acting. These needs drive the retrieval of relevant transition evidence across trajectories, which is then synthesized into training targets that capture decision-oriented dynamics instead of reconstructing the next observation. This aligns the training objective with the dynamics the policy needs before acting, not with the contents of the next observation. Experimental results validate the effectiveness of AAWM across multiple environments and training settings. These results show that decision-aware world-model targets provide a more effective learning signal than next-observation prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。