让机器人异步预想未来画面，实现实时控制与高成功率兼顾。
GlanceWAM: Sparse Test-Time Imagination for World-Action Models
- 异步生成未来帧，不阻塞控制流程
- 在RoboCasa上达72.2%成功率，比同步模型快24倍
- 适合需要快速响应的机器人视觉决策任务
视频生成模型为机器人学习提供丰富的物理先验，但现有世界-动作模型（WAM）面临根本矛盾：同步生成视频延迟过高，放弃测试时视觉想象则降低任务成功率。本文提出GlanceWAM，通过在单个视频DiT中解耦想象与控制，使异步的前景预览在慢时钟下于后台生成未来数秒的画面，而动作头以48毫秒/块的控制速率在潜在空间直接解码动作，无需等待。该方法依赖非干扰注意力掩码隔离视频表示，并采用抗老化时域训练策略应对异步预览的滞后。仅用示范数据训练，其在24任务的RoboCasa厨房基准上达到72.2%的成功率，超过同步模型Cosmos Policy的67.1%和无想象联合训练的64.4%；在LIBERO上达99.0%，且在NVIDIA A100 GPU上每块执行仅需48毫秒，比同步基线快24倍。
原文摘要 · Abstract (English)
Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control within a single video DiT: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (surpassing synchronous Cosmos Policy at 67.1% and imagination-free co-training at 64.4%) and 99.0% on LIBERO, executing at 48 ms per chunk on an NVIDIA A100 GPU (24x faster than synchronous baselines). Code is available at https://github.com/linhanwang/GlanceWAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。