arXiv:2606.19531cs.CVcs.RO2026-06被引 11

用图像编辑替代视频生成,让机器人模型更高效精准

ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?

论文配图:ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
图 1 · 摘自论文原文
  • 用预训练图像编辑模型生成动作上下文,避免视频生成开销
  • 推理速度提升4倍,计算量降为1/6,性能超越基线模型
  • 适合追求低延迟、高精度的机器人控制场景

世界动作模型(WAMs)通常依赖视频生成来连接视觉建模与机器人控制。然而,基于视频的WAMs存在三大耦合问题:密集多帧未来令牌导致推理成本高,完整视频预测浪费资源在与动作无关的时序和外观细节上,长时程未来想象可能引入误差并误导动作预测。这引发一个核心问题:世界动作模型是否真需视频生成?我们提出ImageWAM,一种利用预训练图像编辑模型进行机器人动作预测的简化框架。与视频生成不同,图像编辑提供更匹配的先验:仅需建模目标帧的变换,聚焦于当前到目标的视觉差异,且通过编辑预训练将任务指令锚定在局部视觉变化中。实际应用中,ImageWAM不解码目标帧;而是将图像编辑去噪过程产生的键值缓存作为紧凑的世界-动作上下文,条件化流匹配动作专家。ImageWAM在多个仿真与真实世界实验中,无需额外策略预训练即超越标准VLA基线,并达到与竞争性WAM相当的性能。其计算量降至视频基线的1/6,延迟降低至1/4。注意力分析显示,编辑缓存聚焦于任务相关的变化区域,支持图像编辑作为视频建模的有效替代方案。

原文摘要 · Abstract (English)

World Action Models (WAMs) commonly rely on video generation to bridge visual world modeling and robot control. However, video-based WAMs face three coupled limitations: dense multi-frame future tokens make inference costly, full video prediction spends capacity on action-irrelevant temporal and appearance details, and long-horizon future imagination may introduce errors that mislead action prediction. These issues raise a simple question: Does world action model really need video generation? We propose ImageWAM, a simple WAM framework that repurposes pretrained image editing models for robot action prediction. In contrast to video generation, image editing provides a better-matched prior: it only needs to model a target-frame transformation, focuses on action-relevant current-to-target visual differences, and grounds task instructions to localized visual changes through edit pretraining. In practice, ImageWAM does not decode the target frame at inference time; instead, it conditions a flow-matching action expert on the KV caches produced by image-editing denoising, using them as a compact world-action context. ImageWAM outperforms standard VLA baselines and matching competitive WAMs without additional policy pretraining across different simulator and real-world experiments. It also reduces FLOPs to 1/6 and latency to 1/4 of video-based WAMs. Attention analysis further shows that editing caches focus on task-relevant change regions, supporting image editing as an effective alternative to video-based world-action modeling.

机器人控制图像编辑高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。