让视觉模型先推理运动轨迹，再生成画面，提升预测真实性和机器人学习效率。
FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Models
- 引入视觉思维链，先预测光流再生成下一帧图像。
- 在机器人任务中实现更合理画面预测与更高样本效率。
- 适合关注物理合理性与高效策略训练的研究者。
许多视觉-语言-动作（VLA）模型依赖于基于下一帧预测的内部世界模型（$v_t \rightarrow v_{t+1}$）。然而，这种范式直接预测未来帧外观，缺乏对潜在运动动态的显式推理，常导致物理上不合理的视觉预测和低效策略学习。为此，我们提出视觉思维链（Visual CoT），强制模型在生成未来帧前先推理运动动态。我们构建了FlowVLA，一种自回归Transformer，将推理过程显式表达为 $v_t \rightarrow f_t \rightarrow v_{t+1}$，其中 $f_t$ 为中间光流预测，天然编码运动信息。通过先遵循 $f_t$ 所表达的运动规划，该过程使预训练目标（动态预测）与下游任务（动作生成）自然对齐。我们在挑战性的机器人操作基准及真实机器人上进行实验，结果表明，FlowVLA不仅生成更连贯、物理上更合理的视觉预测，且在策略性能上达到当前最优，并显著提升样本效率，为VLA中的世界建模提供了更严谨的基础。
原文摘要 · Abstract (English)
Many Vision-Language-Action (VLA) models are built upon an internal world model trained via next-frame prediction ``$v_t \rightarrow v_{t+1}$''. However, this paradigm attempts to predict the future frame's appearance directly, without explicitly reasoning about the underlying dynamics. \textbf{This lack of an explicit motion reasoning step} often leads to physically implausible visual forecasts and inefficient policy learning. To address this limitation, we introduce the \textbf{Visual Chain of Thought (Visual CoT)}, a paradigm that compels the model to first reason about \textbf{motion dynamics} before generating the future frame. We instantiate this paradigm by proposing \textbf{FlowVLA}, an autoregressive Transformer that explicitly materializes this reasoning process as ``$v_t \rightarrow f_t \rightarrow v_{t+1}$'', where $f_t$ is an intermediate optical flow prediction that inherently encodes motion. By forcing the model to first follow the motion plan encoded by $f_t$, this process inherently \textbf{aligns the pre-training objective of dynamics prediction with the downstream task of action generation.} We conduct experiments on challenging robotics manipulation benchmarks, as well as real-robot evaluations. Our FlowVLA not only generates \textbf{more coherent and physically plausible visual predictions}, but also achieves state-of-the-art policy performance with \textbf{substantially improved sample efficiency}, pointing toward a more principled foundation for world modeling in VLAs. Project page: https://irpn-lab.github.io/FlowVLA/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。