arXiv:2511.14262cs.LGcs.AI2025-11AAAI被引 4

用物体中心的注意力机制建模环境,让强化学习更懂因果关系。

Object-Centric World Models for Causality-Aware Reinforcement Learning

论文配图:Object-Centric World Models for Causality-Aware Reinforcement Learning
图 1 · 摘自论文原文
  • 用物体中心的Transformer构建世界模型,分解环境为可追踪的物体
  • 在3个物体丰富任务上,样本效率和最终表现均优于现有方法
  • 适合需要理解复杂交互的智能体决策场景

世界模型旨在支持样本高效的深度强化学习。然而,对于高维、非平稳且由多个物体及复杂交互构成的环境,现有世界模型因学习整体表征而难以准确模拟。人类则通过将环境分解为离散物体来高效决策。受此启发,我们提出STICA框架:物体中心的Transformer作为世界模型,同时用于策略与价值网络。该框架将每帧观测表示为一组物体中心的令牌,外加动作与奖励令牌,使世界模型能预测各令牌的动态与交互。策略与价值网络在注意力层中估计令牌间的因果关系,实现因果引导的决策。在多个物体丰富的基准测试中,STICA在样本效率和最终性能上均持续优于当前最优代理。

原文摘要 · Abstract (English)

World models have been developed to support sample-efficient deep reinforcement learning agents. However, it remains challenging for world models to accurately replicate environments that are high-dimensional, non-stationary, and composed of multiple objects with rich interactions since most world models learn holistic representations of all environmental components. By contrast, humans perceive the environment by decomposing it into discrete objects, facilitating efficient decision-making. Motivated by this insight, we propose \emph{Slot Transformer Imagination with CAusality-aware reinforcement learning} (STICA), a unified framework in which object-centric Transformers serve as the world model and causality-aware policy and value networks. STICA represents each observation as a set of object-centric tokens, together with tokens for the agent action and the resulting reward, enabling the world model to predict token-level dynamics and interactions. The policy and value networks then estimate token-level cause--effect relations and use them in the attention layers, yielding causality-guided decision-making. Experiments on object-rich benchmarks demonstrate that STICA consistently outperforms state-of-the-art agents in both sample efficiency and final performance.

强化学习因果推理物体中心世界模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。