arXiv:2607.25236cs.CLcs.RO2026-07

用代码形式构建可规划的世界模型,自动学习动态规律并支持实时重规划。

VisualPatchWorld: Code World Models as Latent Structured Representations for Planning

论文配图:VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
图 1 · 摘自论文原文
  • 通过短探针选择动态形式,再拟合参数实现代码化世界模型
  • 规划成功率达69.0%,比最强代码基线高23.5个百分点
  • 适合需要精确动态建模的机器人控制与自主规划场景

不同研究将“世界模型”用于不同目的,但核心目标一致:以支持感知、模拟和规划的形式捕捉动作下的世界演化。现有方法主要有两类:基于神经网络的连续向量空间动态预测器,以及显式状态与物理定律的手工物理引擎。前者数据驱动但动态隐含,后者可解释但难以规模化构建。本文提出视觉补丁世界(VisualPatchWorld, VPW),将世界动态表示为代码。VPW首先通过短探针选定定性动态形式,再从记录的状态-动作轨迹中最小化多步预测误差来拟合该形式的自由参数。生成的程序可如模拟器般推进,以源码形式可查,亦可用于模型预测控制;图像导出的场景图可在重规划时提供实时状态。在与先前代码型世界模型的对比中,VPW达到69.0%平均规划成功率,超越最强代码基线23.5个百分点。当正确选择定性动态至关重要时,提升最为显著。在相同规划器下,所建模型在导航与抓取密集控制任务中逼近真实引擎表现;接触密集推移任务仍存残差,但通过在引擎中检查一组候选计划,可弥补大部分差距。结果表明,自动构建的代码世界模型可有效用于规划。代码已开源:https://github.com/HKBU-KnowComp/VisualPatchWorld/

原文摘要 · Abstract (English)

Different research lines use the term world model in different ways, yet they share a common aim: to capture how the world evolves under action in a form that supports perception, simulation, and planning. Two prominent realizations are neural predictors that learn dynamics in continuous vector spaces, and hand-built physics engines that expose explicit state and physical laws. Neural predictors scale from data but leave the form of the dynamics implicit; physics engines are inspectable and editable but difficult to construct at scale. We introduce VisualPatchWorld (VPW), which represents world dynamics as code. VPW first selects a qualitative dynamical form with short active probes, then fits that form's free parameters from recorded state-action traces by minimizing multi-step prediction error. The resulting programs can be rolled forward like a simulator, inspected in source form, and used inside model-predictive control; image-derived scene graphs can supply the live state at replan time. Across comparisons with prior code-based world models, VPW attains 69.0% mean planning success and exceeds the strongest code baseline by 23.5 points. The largest gains arise when choosing the correct qualitative dynamics is essential. Under the same planner, the induced models approach ground-truth engine success on navigation and grasp-rich control; a residual gap remains for contact-rich pushing, and checking a shortlist of promising plans in the engine closes most of that gap. These results establish a practical route toward automatically constructed code world models that are useful for planning. Code is available at https://github.com/HKBU-KnowComp/VisualPatchWorld/.

世界模型代码表示规划机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。