arXiv:2507.13231cs.CVcs.AI2025-07被引 21

无需条件输入和迭代去噪,直接从视觉映射动作,提速一倍以上。

VITA: Vision-to-Action Flow Matching Policy

  • 跳过传统条件模块,直接由视觉特征生成动作向量。
  • 在9个仿真与5个真实任务上推理速度提升1.5至2倍。
  • 适合追求高效实时控制的机器人应用,尤其看重推理速度的场景。

传统流匹配与基于扩散的策略通过从标准噪声分布(如高斯分布)迭代去噪进行采样,并需在生成过程中反复引入视觉信息的条件模块,导致显著的时间与内存开销。为降低复杂性,我们提出VITA(视觉到动作流匹配策略),一种无噪声、无条件的流匹配策略学习框架,可直接从视觉表征流向潜在动作。由于流的起点具有视觉基础,生成阶段无需视觉条件。然而,视觉与动作之间的桥接极具挑战:动作维度更低、结构更稀疏,且流匹配要求源与目标维度一致。为此,我们引入动作自编码器,将原始动作映射到与视觉隐空间对齐的结构化潜在空间,并与流匹配联合训练。为防止端到端训练中潜在动作空间坍缩,我们提出流潜在解码机制,通过将动作重构损失反向传播至流匹配常微分方程求解步骤来锚定生成过程。我们在ALOHA与Robomimic的9个仿真任务和5个真实世界任务上评估了VITA,结果表明其推理速度相比含条件模块的方法提升1.5至2倍,同时性能优于或媲美当前最优策略。

原文摘要 · Abstract (English)

Conventional flow matching and diffusion-based policies sample via iterative denoising from standard noise distributions (e.g., Gaussian), and require conditioning modules to repeatedly incorporate visual information during the generative process, incurring substantial time and memory overhead. To reduce the complexity, we develop VITA, VIsion-To-Action policy, a noise-free and conditioning-free flow matching policy learning framework that directly flows from visual representations to latent actions. Since the source of the flow is visually grounded, VITA eliminates the need for visual conditioning during generation. As expected, bridging vision and action is challenging, because actions are lower-dimensional, less structured, and sparser than visual representations; moreover, flow matching requires the source and target to have the same dimensionality. To overcome this, we introduce an action autoencoder that maps raw actions into a structured latent space aligned with visual latents, trained jointly with flow matching. To further prevent latent action space collapse during end-to-end training, we propose flow latent decoding, which anchors the latent generation process by backpropagating the action reconstruction loss through the flow matching ODE (ordinary differential equation) solving steps. We evaluate VITA on 9 simulation and 5 real-world tasks from ALOHA and Robomimic. VITA achieves 1.5x-2x faster inference compared to conventional methods with conditioning modules, while outperforming or matching state-of-the-art policies. Project page: https://ucd-dare.github.io/VITA/.

机器人控制流匹配动作生成高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。