arXiv:2603.10712cs.RO2026-03被引 7

让机器人提前预判动作与视觉变化,提升智能体决策能力。

FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model

  • 分离视觉与动作信息,联合建模物理规律
  • 在多数据集上显著提升视觉-语言-动作模型性能
  • 适合需要精准动作预测的机器人研究者

预测性前瞻对具身智能体至关重要。由于机器人运动受其视觉感知的环境几何结构制约,有效预判未来需捕捉视觉与动作间的紧密耦合关系。现有视觉-语言-动作模型虽尝试引入未来引导,但在联合建模方面表现不佳:显式方法过度关注无关视觉细节,隐式方法依赖稀疏帧对破坏时间连续性。这些方法因过度依赖视觉重建而变得视觉主导,混淆静态场景与动态动作意图。我们提出FutureVLA,采用新型联合视觉-动作预测架构,通过解耦视觉与动作信息,再联合编码通用物理先验,实现更优的联合建模。预训练阶段,利用异构操作数据集并引入联合视觉-动作门控机制,结构化分离视觉状态保持与时间动作建模,使动作流专注连续物理动态,同时显式查询视觉令牌获取环境约束,生成高度泛化的联合视觉-动作嵌入。后训练阶段,采用潜在嵌入对齐策略,使多种下游VLA模型可内化这些时间先验,无需修改推理架构。大量实验证明,FutureVLA持续提升现有VLA框架性能。

原文摘要 · Abstract (English)

Predictive foresight is important to intelligent embodied agents. Since the motor execution of a robot is intrinsically constrained by its visual perception of environmental geometry, effectively anticipating the future requires capturing this tightly coupled visuomotor interplay. While recent vision-language-action models attempt to incorporate future guidance, they struggle with this joint modeling. Existing explicit methods divert capacity to task-irrelevant visual details, whereas implicit methods relying on sparse frame pairs disrupt temporal continuity. By heavily relying on visual reconstruction, these methods become visually dominated, entangling static scene context with dynamic action intent. We argue that effective joint visuomotor predictive modeling requires both temporal continuity and visually-conditioned supervision decoupling. To this end, we propose FutureVLA, featuring a novel Joint Visuomotor Predictive Architecture. FutureVLA is designed to extract joint visuomotor embeddings by first decoupling visual and motor information, and then jointly encoding generalized physical priors. Specifically, in the pretraining stage, we leverage heterogeneous manipulation datasets and introduce a Joint Visuomotor Gating mechanism to structurally separate visual state preservation from temporal action modeling. It allows the motor stream to focus on continuous physical dynamics while explicitly querying visual tokens for environmental constraints, yielding highly generalizable joint visuomotor embeddings. Subsequently, in the post-training stage, we employ a latent embeddings alignment strategy, enabling diverse downstream VLA models to internalize these temporal priors without modifying their inference architectures. Extensive experiments demonstrate that FutureVLA consistently improves VLA frameworks.

视觉-动作预测机器人学习具身智能多模态建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。