用物体中心表示实现可规划的机器人控制,适应新场景更高效。
Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations

- 通过视觉编码器学习物体级表征,构建动作条件的动力学模型。
- 在模拟任务中,任务成功率显著高于非物体中心基线模型。
- 适合需要灵活决策的机器人操控场景,尤其擅长处理未见情况。
预测性世界模型使智能体能够建模场景动态并推理自身行为后果。受人类感知启发,物体中心世界模型使用物体级表示捕捉场景动态,可用于下游任务如动作规划。然而,大多数物体中心模型和强化学习方法学习的是推理时固定的反应式策略,限制了对新情境的泛化能力。本文提出 Slot-MPC,一种基于物体中心表示的世界建模框架,支持通过模型预测控制(MPC)进行规划。Slot-MPC 利用视觉编码器学习基于槽位的表征,编码场景中的各个物体,并利用这些结构化表征训练动作条件的物体中心动力学模型。推理时,该动力学模型支持通过 MPC 进行动作规划,使智能体能适应先前未见的情境。由于世界模型可微分,可使用基于梯度的 MPC 直接优化动作,相比依赖采样的无梯度方法计算更高效。在模拟机器人操作任务上的实验表明,相比于非物体中心基线模型,Slot-MPC 在任务性能和规划效率上均有提升。在状态-动作覆盖有限的离线设置下,基于梯度的 MPC 表现优于无梯度采样方法。结果表明,显式结构化的物体中心表征为可控且可泛化的决策提供了强归纳偏置。代码与附加结果见 https://slot-mpc.github.io。
原文摘要 · Abstract (English)
Predictive world models enable agents to model scene dynamics and reason about the consequences of their actions. Inspired by human perception, object-centric world models capture scene dynamics using object-level representations, which can be used for downstream applications such as action planning. However, most object-centric world models and reinforcement learning (RL) approaches learn reactive policies that are fixed at inference time, limiting generalization to novel situations. We propose Slot-MPC, an object-centric world modeling framework that enables planning through Model Predictive Control (MPC). Slot-MPC leverages vision encoders to learn slot-based representations, which encode individual objects in the scene, and uses these structured representations to learn an action-conditioned object-centric dynamics model. At inference time, the learned dynamics model enables action planning via MPC, allowing agents to adapt to previously unseen situations. Since the learned world model is differentiable, we can use gradient-based MPC to directly optimize actions, which is computationally more efficient than relying on gradient-free, sampling-based MPC methods. Experiments on simulated robotic manipulation tasks show that Slot-MPC improves both task performance and planning efficiency compared to non-object-centric world model baselines. In the considered offline setting with limited state-action coverage, we find that gradient-based MPC performs better than gradient-free, sampling-based MPC. Our results demonstrate that explicitly structured, object-centric representations provide a strong inductive bias for controllable and generalizable decision-making. Code and additional results are available at https://slot-mpc.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。