arXiv:2605.06222cs.ROcs.AI2026-05被引 12

让机器人根据想象与现实的吻合度,自动调整下一步动作长度。

When to Trust Imagination: Adaptive Action Execution for World Action Models

论文配图:When to Trust Imagination: Adaptive Action Execution for World Action Models
图 1 · 摘自论文原文
  • 用多模态注意力机制实时验证想象未来是否可信
  • 实测减少69%模型推演次数,成功率提升35%
  • 适合需要长程规划又怕出错的复杂操作场景

世界动作模型(WAMs)通过联合预测未来视觉和动作,成为机器人操作的新范式。但现有方法在每次推理后固定执行若干动作,无法感知想象与真实之间的偏差。本文将自适应执行建模为未来-现实一致性验证问题:当预测可靠时延长执行,偏离时提前重规划。提出轻量级验证器FFDC,联合分析预测动作、视觉动态、真实观测与语言指令,判断剩余动作序列是否可信。该机制使动作块大小自适应生成,在保持长程效率的同时恢复对接触密集或困难阶段的响应能力。进一步引入混合时域训练,提升长程轨迹覆盖。在RoboTwin基准和真实世界实验中,该方法显著提升鲁棒性与效率:在RoboTwin上减少69.10%的前向传播,执行时间降低34.02%,成功率提升2.54%;真实场景中成功率提高35%。

原文摘要 · Abstract (English)

World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation by jointly predicting future visual observations and future actions. However, current WAMs typically execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the actual physical rollout. In this work, we formulate adaptive WAM execution as a future-reality verification problem: the robot should execute longer when the WAM-predicted future remains reliable, and replan earlier when reality deviates from imagination. To this end, we propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that jointly reasons over predicted future actions, predicted visual dynamics, real observations, and language instructions to estimate whether the remaining action rollout can still be trusted. FFDC enables adaptive action chunk sizes as an emergent consequence of prediction-observation consistency, preserving the efficiency of long-horizon execution while restoring responsiveness in contact-rich or difficult phases. We further introduce Mixture-of-Horizon Training to improve long-horizon trajectory coverage for adaptive execution. Experiments on the RoboTwin benchmark and in the real world demonstrate that our method achieves a strong robustness-efficiency trade-off: on RoboTwin, it reduces WAM forward passes by 69.10% and execution time by 34.02%, while improving success rate by 2.54% over the short-chunk baseline; in real-world experiments, it improves success rate by 35%.

机器人操作自适应执行动作模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。