提出分层因果潜状态机，让机器人模型更懂物体分离与因果关系。
HCLSM: Hierarchical Causal Latent State Machines for Object-Centric World Modeling
- 用槽注意力分解物体，分三层建模时间动态
- 在PushT任务上实现0.008的预测误差，空间分解损失0.0075
- 适合需要理解物理与事件结构的机器人控制场景
现有世界模型依赖扁平潜表示，导致物体混淆、忽略因果结构、时间动态单一。本文提出HCLSM，基于三个核心原则：通过槽注意力与空间广播解码实现物体中心分解；构建三层次时序引擎，结合选择性状态空间模型（用于连续物理）、稀疏变压器（用于离散事件）和压缩变压器（用于抽象目标）；通过图神经网络学习交互模式实现因果结构建模。采用两阶段训练：先通过空间重建强制槽专一化，再进行动态预测。在Open X-Embodiment数据集的PushT机器人操控基准上训练6800万参数模型，实现0.008的下一状态预测均方误差，空间分解损失为0.0075，且能自发学习事件边界。自定义Triton内核使SSM扫描速度提升38倍。系统共8478行Python代码，覆盖51个模块，含171个单元测试。
原文摘要 · Abstract (English)
World models that predict future states from video remain limited by flat latent representations that entangle objects, ignore causal structure, and collapse temporal dynamics into a single scale. We present HCLSM, a world model architecture that operates on three interconnected principles: object-centric decomposition via slot attention with spatial broadcast decoding, hierarchical temporal dynamics through a three-level engine combining selective state space models for continuous physics, sparse transformers for discrete events, and compressed transformers for abstract goals, and causal structure learning through graph neural network interaction patterns. HCLSM introduces a two-stage training protocol where spatial reconstruction forces slot specialization before dynamics prediction begins. We train a 68M-parameter model on the PushT robotic manipulation benchmark from the Open X-Embodiment dataset, achieving 0.008 MSE next-state prediction loss with emerging spatial decomposition (SBD loss: 0.0075) and learned event boundaries. A custom Triton kernel for the SSM scan delivers 38x speedup over sequential PyTorch. The full system spans 8,478 lines of Python across 51 modules with 171 unit tests. Code: https://github.com/rightnow-ai/hclsm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。