提出高效未来条件化方法,让机器人模型更准更快地应对环境变化。
Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models

- 用稀疏机制只在关键阶段计算未来信息,减少重复计算。
- 在分布式测试集上成功率从49.14%提升至73.57%,速度比联合模型快2.21倍。
- 适合追求推理效率与鲁棒性平衡的机器人控制研究者。
世界动作模型(WAMs)通过学习当前观测之外的环境演化,提升机器人操作能力。然而现有方法面临根本矛盾:联合式WAMs在推理时保留未来感知表征,但计算开销巨大;高效替代方案则在推理时移除未来建模,可能丧失时序推理带来的鲁棒性优势。本文重新审视未来表征的作用,证明推理时未来条件化对分布外泛化至关重要。为此提出Faster-WAM,一种高效未来条件化框架,在保留未来表征的同时避免昂贵的视频-动作交互。Faster-WAM引入稀疏未来条件化机制,仅一次计算未来表征并选择性复用以完成动作去噪。具体提出SparseMoT,以选择性视频-动作交互替代全层融合;以及Interval KV-Fusion,无需增加注意力复杂度即可聚合多深度未来表征。实验表明,Faster-WAM在性能-效率权衡上显著优于现有WAMs。在分布外的LIBERO-Plus基准上,成功率从49.14%提升至73.57%,运行速度比联合式模型快2.21倍。同时在LIBERO和RoboTwin 2.0上达到最新水平,并展现出强大的真实场景操作鲁棒性。
原文摘要 · Abstract (English)
World Action Models (WAMs) improve robot manipulation by learning how the environment evolves beyond the current observation. However, existing approaches face a fundamental dilemma: Joint-WAMs preserve future-aware representations during inference but incur prohibitive computation costs, while efficient alternatives remove future modeling at inference time and may lose the robustness benefits of temporal reasoning. In this work, we revisit the role of future representations in WAMs and show that inference-time future conditioning is critical for generalization under distribution shifts. This observation motivates Faster-WAM, an efficient future-conditioning WAM that preserves future representations while avoiding expensive video-action interaction. Faster-WAM introduces a sparse future-conditioning framework that computes future representations once and selectively reuses them throughout action denoising. Specifically, we propose SparseMoT to replace ubiquitous layer-wise fusion with selective video-action interaction at a compact subset of network stages, and Interval KV-Fusion to aggregate multi-depth future representations without increasing attention complexity. Experiments demonstrate that Faster-WAM achieves a substantially better performance-efficiency trade-off than existing WAMs. On the out-of-distribution LIBERO-Plus benchmark, Faster-WAM improves success rate from 49.14% to 73.57% compared with Fast-WAM, while running 2.21$\times$ faster than Joint-WAM. It further achieves state-of-the-art performance on LIBERO and RoboTwin 2.0, while demonstrating strong robustness in real-world manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。