arXiv:2606.09811cs.ROcs.AI2026-06被引 16

让视觉和动作分步运行,提升机器人操控的效率与精度。

AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing

论文配图:AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing
图 1 · 摘自论文原文
  • 视觉与动作用不同频率运行,视觉长期规划,动作实时执行。
  • 在真实任务中达78.3%成功率,闭环控制速度提升4.59倍。
  • 适合需要快速响应的机器人操控场景,尤其无预训练数据时。

世界-动作模型作为机器人操作的新范式,联合建模视觉场景动态与动作,将物理先验注入策略学习。然而现有模型强制视觉与动作在同一时间分辨率下耦合,导致视觉分支需处理冗余且信息量低的短期帧变化。本文提出AHA-WAM,一种基于双扩散变换器(DiT)架构的异步时序自适应世界-动作模型,围绕这一时间不对称性重构建模方式。其中,视频DiT作为低频世界规划器,通过滚动键值记忆保存过往观测,并输出可复用的层间潜在上下文以表征长时序场景演化;动作DiT则通过层间联合注意力查询该上下文,以高频方式执行短周期动作块。为支持异步执行,引入时序自适应偏移训练与观察引导的视频上下文路由(OVCR),使动作专家能利用长时序世界上下文,同时保持对实时执行状态的响应能力,无需重跑视频DiT。在RoboTwin和真实世界操作任务上的实验表明,AHA-WAM无需任何机器人数据预训练,即达到92.80%的平均成功率(RoboTwin),4个真实任务综合成功率78.3%,并实现24.17 Hz闭环控制,较Fast-WAM提速4.59倍。

原文摘要 · Abstract (English)

World-action models have emerged as a promising paradigm for robot manipulation, jointly modeling visual scene dynamics and actions to inject physical priors into policy learning. However, existing world-action models couple world prediction and action execution at the same temporal resolution, forcing the world branch to model near-term frame variations that are redundant and weakly informative. We posit that strictly binding world prediction and action execution to the same temporal rhythm may underutilize the potential of the video branch for embodied control. Therefore, we propose AHA-WAM, an Asynchronous Horizon-Adaptive World-Action Model built on a dual Diffusion Transformer (DiT) architecture that reorganizes world-action modeling around this temporal asymmetry. AHA-WAM instantiates the video DiT as a low-frequency world planner that maintains rolling key-value memory over past observations and exposes reusable layerwise latent context encoding long-horizon scene evolution, while a high-frequency action DiT executes short action chunks in closed loop by querying this context through layerwise joint attention. To support asynchronous execution, we introduce horizon-adaptive offset training and Observation-Guided Video-Context Routing (OVCR), which together let the action expert exploit long-horizon world context while remaining responsive to real-time execution state without rerunning the video DiT. Experiments on RoboTwin and real-world manipulation tasks show that AHA-WAM achieves state-of-the-art performance without any robot-data pretraining, attaining 92.80% average success on RoboTwin and 78.3% success across 4 real-world tasks, while reaching 24.17 Hz closed-loop control with a 4.59x speedup over Fast-WAM.

机器人操控扩散模型异步执行长时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。