用可学习的算子结构显式建模潜空间演化,提升机器人动作模型性能。
Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models
- 基于算子传播与强制项的显式演化建模,替代传统Transformer结构。
- 在两个不同架构中均提升闭环表现与鲁棒性,支持全替换仍有效。
- 适合关注潜空间动态建模与机器人控制的开发者和研究者。
世界动作模型(WAMs)通过预测任务相关场景状态在交互下的演化来增强机器人策略。近期的WAMs越来越多地在潜表示空间中进行预测,避免全量外观生成的同时保留控制相关的信息。然而,潜空间演化通常由基于Transformer的预测器实现,其归纳偏置集中于标记间交互而非时间演化。本文将演化实现视为独立于预测表示与预测-策略耦合的架构选择。提出潜演化算子网络(LEON),通过上下文调制的算子传播与加性强制项,在可学习的可观测空间中建模潜空间演化。基于受控Koopman生成器视角,LEON围绕共享演化算子结构组织上下文依赖的演化变化,同时保留加性变化路径。受控动力系统验证了所得演化特定归纳偏置及算子传播与强制项的互补作用。在两种集成潜空间预测到策略中的WAM框架下,LEON提升了闭环性能与鲁棒性,且在完全替换演化模块时仍保持有效性。结果表明,演化实现是潜空间WAM中一个关键的架构选择。
原文摘要 · Abstract (English)
World Action Models (WAMs) augment robot policies by predicting how task-relevant scene states may evolve under interaction. Recent WAMs increasingly perform such prediction in latent representation spaces, avoiding full appearance-level generation while preserving control-relevant information. Yet latent transitions are commonly realized with Transformer-based predictors whose inductive structure is centered on token interaction rather than temporal evolution. We study transition realization as an architectural choice distinct from predictive representation and prediction-policy coupling. We introduce the Latent Evolution Operator Network (LEON), which models latent evolution in a learned observable space through context-modulated operator-based propagation and additive forcing. Grounded in the controlled Koopman generator view of evolution, LEON organizes context-dependent transition variation around a shared evolution-operator structure while retaining a complementary path for additive change. Controlled dynamical systems verify the resulting evolution-specific inductive bias and the complementary roles of operator propagation and forcing. Across two WAM formulations that integrate latent prediction into the policy differently, LEON improves closed-loop performance and robustness while remaining effective under full transition replacement. These results establish transition realization as a consequential architectural choice in latent WAMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。