arXiv:2607.18715cs.AI2026-07被引 1

将环境变化与动作影响分离,提升模型对世界动态的建模能力。

DWM: Separating World Effects from Actions in Latent World Models

论文配图:DWM: Separating World Effects from Actions in Latent World Models
图 1 · 摘自论文原文
  • 设计双头结构,显式分解状态变化为动作相关与动作无关两部分。
  • 在三个新构建的带持续环境效应任务上,规划成功率平均提升13.1%。
  • 无需改动原有架构,适合需要高可迁移性的强化学习场景。

隐变量世界模型是现代基于模型控制的核心,但现有动作条件化方法以单一、混合的目标监督下一时刻隐状态转移,使所有状态变化源混杂在一起。现实中,状态变化来自两类异质来源:由智能体动作驱动的部分,以及不依赖动作的环境固有动态(如重力滑动、惯性、接触反弹和持续漂移)。将二者融合为单一目标会混淆因果关系,损害模型泛化能力。本文提出DWM(分解世界模型),在监督层面实现分离:在原预测头外增加一个辅助世界头,通过归一化世界对比损失使其保持动作不变性,并与原头施加正交约束;二者共同诱导出显式的加法分解,预测转移分为动作不变与互补的动作驱动成分,不改变底层架构或推理流程。为评估持续环境效应下的表现,构建了三个标准基准的W变体——PushT-W、Reacher-W、TwoRoom-W,各自模拟不同动作不变动态。DWM在平坦基准上媲美强基线,在三类W变体上平均提升CEM规划成功率13.1%。

原文摘要 · Abstract (English)

Latent world models underpin much of modern model-based control, yet current action-conditioned formulations supervise the next-latent transition with a single, undifferentiated target, forcing a monolithic learning signal to absorb every source of state change. In real world, however, transitions arise from two heterogeneous sources: an action-driven component induced by the agent, and an action-invariant world effect -- the change that would still occur under a null action, dictated by the environment's intrinsic dynamics (e.g., gravity-driven sliding, inertia, contact rebound, and persistent drift). Fusing them into a single target entangles the two inside the latent transition, prevents the model from attributing observed changes to their underlying causes, and undermines the transferability of the learned dynamics. We introduce DWM (Decomposed World Model), a supervision-level framework that operationalizes this decomposition. DWM augments the predictor of a latent world model with an auxiliary world head, regularized by a normalized world-contrastive objective to be action-invariant, while the original pred head is coupled to it via an orthogonality constraint; together, the two signals induce an explicit additive decomposition of the predicted transition into an action-invariant and a complementary action-driven component, without altering the underlying architecture or inference pipeline. To evaluate DWM under persistent world effects, we construct W-variants of three standard control benchmarks -- PushT-W, Reacher-W, and TwoRoom-W -- each instantiating a distinct action-invariant dynamic. DWM matches strong baselines on the flat counterparts and delivers a mean absolute improvement of 13.1% in CEM planning success across the W-variants.

世界模型分解建模强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。