arXiv:2608.02365cs.AIcs.LG2026-08被引 3

轻量动作头搭配深度视频模型,实现快速高精度机器人动作预测。

Faster-WAM: Do World Action Models Need Deep Action Modules?

论文配图:Faster-WAM: Do World Action Models Need Deep Action Modules?
图 1 · 摘自论文原文
  • 视频中心设计,用轻量头对接30层视频骨干,降低计算开销。
  • 单次推理仅需66.5毫秒,比Fast-WAM快3.2倍,性能相当。
  • 无需额外预训练,在多个数据集上表现稳健,适合实际部署。

世界动作模型(WAMs)将机器人动作预测与视频世界模型结合。现有方法常使动作模块深度与视频骨干一致,导致计算开销大、推理延迟高。本文提出视频中心设计原则Dock of Transformer(DoT),将预训练视频Transformer作为表示枢纽,通过对接接口连接轻量输出头,支持灵活设计并直接访问骨干所有层的特征。基于此,我们提出Faster-WAM,将其单层动作头对接30层视频骨干,对接接口融合所有视频层的键值,并应用RoPE重对齐。无需额外具身预训练,Faster-WAM在LIBERO和RoboTwin 2.0上表现优异,且在LIBERO-Plus上展现强分布外泛化能力。在控制对比中,其端到端延迟最低,单次推理仅需66.5毫秒,较Fast-WAM提速3.2倍。结果表明,该架构兼顾任务定制性、低延迟、高性能与鲁棒泛化。

原文摘要 · Abstract (English)

World Action Models (WAMs) couple robot action prediction with video world models. Existing WAMs with shared-backbone and Mixture-of-Transformers designs generally tie the depth of the action module to that of the video backbone, resulting in substantial computational overhead and high inference latency. To address this limitation, we introduce Dock of Transformer (DoT), a video-centric design principle that treats a pretrained video Transformer as a representation hub and connects lightweight output-heads through docking interfaces. This enables flexible output-head design while providing direct access to representations from all layers of the backbone. We then introduce \textbf{Faster-WAM}, an instantiation of DoT for WAMs, which docks a single-layer action head onto a 30-layer video backbone. The docking interface fuses keys and values from all video layers and applies RoPE realignment. Without additional embodied pretraining, Faster-WAM achieves competitive performance on LIBERO and RoboTwin 2.0 while demonstrating strong out-of-distribution generalization on LIBERO-Plus. Faster-WAM also achieves the lowest end-to-end latency in our controlled comparison, requiring only 66.5 ms per inference --- a \(3.2\times\) speedup over Fast-WAM. Overall, these results demonstrate that the video-centric DoT architecture supports flexible task-specific head design while delivering low inference latency, strong action-prediction performance, and robust generalization.

动作预测视频建模轻量化机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。