让世界模型模块化,换智能体不用重训环境。
BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Models

- 将环境与智能体分开建模,通过隐式接口连接。
- 冻结背景模型后,跨智能体复用性能不变。
- 适合需要快速更换智能体的连续控制场景。
基于模型的强化学习在连续控制任务中表现优异,依赖于潜在世界模型。然而现有方法多采用整体化的潜在动态建模,将环境动态耦合在一起,导致可复用性差:只要更换智能体就必须从头训练整个世界模型,即使环境不变。为此,我们提出BRICKS-WM(通过接口组合动力学构建可复用结构化世界模型),实现结构化世界模型的模块化组装。基于物理世界由独立实体构成的洞察,我们认为全局动态可分解为若干独立的动力学模块,通过潜在接口交互。作为最小实例,我们将潜在状态空间分解为受控智能体模块与外部背景模块,二者通过学习得到的潜在接口相连。不同于以往以视觉分割为核心的物体中心方法,BRICKS-WM强调转移动态的功能分离,确保背景动态对智能体动态保持无关。实验证明,从零训练时,其控制性能可媲美强基线的单体模型;更关键的是,冻结背景模型后可在不同智能体间复用。
原文摘要 · Abstract (English)
Model-based Reinforcement Learning (MBRL) has achieved remarkable success in continuous control by leveraging latent world models. However, prevailing approaches typically rely on monolithic latent dynamics, entangling environment dynamics into a coupled process. This coupling severely limits reusability: altering the agent necessitates retraining the entire world from scratch, even if the environment remains constant. To address this, we introduce BRICKS-WM (Building Reusability via Interface Composition Kinetics for Structured World Models), a framework for the modular assembly of structured world models. Driven by the insight that the physical world is composed of independent entities, we posit that global dynamics can be modeled as a composition of distinct dynamical modules interacting via latent interfaces. As a minimal instantiation, we factorize the latent state space into an actuated Agent module and an external Background module, bridged by a learned latent interface. Unlike prior object-centric methods that prioritize visual segmentation, BRICKS-WM enforces a functional separation in transition dynamics, ensuring that background dynamics remains agnostic to the agent's dynamics. Empirically, BRICKS-WM achieves control performance comparable to strong monolithic baselines when trained from scratch, and enables the reuse of frozen background dynamics across agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。