用状态空间模型学环境因果关系，比Transformer更优。
Learning Local Causal World Models with State Space Models and Attention
- 用状态空间模型捕捉环境动态，同时发现因果结构。
- 在简单环境中性能相当或更好，且能学习因果表示。
- 适合研究因果推理与高效建模的学者参考。
世界建模是智能体理解物理世界演化规律的核心能力。尽管现有方法表现优异，但大多未能学习环境的因果表示，难以支持复杂任务。本文探讨状态空间模型（SSM）在因果发现方面的潜力，相比广泛使用的Transformer，SSM具有多项优势。实验表明，在相同条件下，等效的SSM模型不仅能准确建模简单环境的动力学，还能同时学习到因果结构，性能与传统Transformer相当甚至更优，为未来基于SSM的因果感知建模研究提供了新方向。
原文摘要 · Abstract (English)
World modelling, i.e. building a representation of the rules that govern the world so as to predict its evolution, is an essential ability for any agent interacting with the physical world. Despite their impressive performance, many solutions fail to learn a causal representation of the environment they are trying to model, which would be necessary to gain a deep enough understanding of the world to perform complex tasks. With this work, we aim to broaden the research in the intersection of causality theory and neural world modelling by assessing the potential for causal discovery of the State Space Model (SSM) architecture, which has been shown to have several advantages over the widespread Transformer. We show empirically that, compared to an equivalent Transformer, a SSM can model the dynamics of a simple environment and learn a causal model at the same time with equivalent or better performance, thus paving the way for further experiments that lean into the strength of SSMs and further enhance them with causal awareness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。