arXiv:2605.30880cs.CLcs.AI2026-05被引 1

不依赖梯度,用代码修复构建可执行世界模型,提升智能体决策能力。

PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments

论文配图:PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments
图 1 · 摘自论文原文
  • 通过反例引导修复代码,将轨迹转为可运行的Python世界模型。
  • 在7个环境上达到76.4%的决策成功率,媲美大模型但无需调用LLM。
  • 揭示观察保真与动作判别力之间的权衡,适合关注可解释性的研究者。

交互式文本智能体的世界模型通常只能从观测-动作轨迹中学习,环境仅返回文本观测,不提供真实隐状态或可检视的转移模型。如何在黑箱设置下生成可执行代码形式的世界模型以支持预测和决策,仍存在研究空白。本文提出PatchWorld,一种无梯度框架,通过反例引导的代码修复,将离线轨迹转化为可执行的Python世界模型。不同于黑箱预测下一观测,PatchWorld生成可检查、可重放、可局部修补的符号化信念状态程序。在七个AgentGym环境中,PatchWorld-Simple在实时单步前瞻中的代码决策得分最高(宏平均成功率达76.4%),与基于LLM的前瞻方法相当,且预测模块内不调用任何LLM。进一步发现,人工指定的残差记忆偏置虽提升观测保真度,却削弱了决策效用,揭示了可执行世界模型中观察保真与动作判别动态间的权衡关系。代码已开源。

原文摘要 · Abstract (English)

World models for interactive text agents must typically be learned from observation-action trajectories alone. Specifically, the environment returns text observations after each action, but does not expose a ground-truth latent state nor an inspectable transition model.A research gap remains in how to induce executable code as a world model in this black-box setting for prediction and agent decision making. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair.Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based decision-making score among evaluated methods (76.4% macro success in live one-step lookahead), matching or exceeding LLM-based lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens agent decision-making utility. This reveals a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.

世界模型代码生成智能体决策可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。