arXiv:2602.09849cs.RO2026-02被引 19

让机器人通过语言与视觉协同规划动作,提升复杂任务执行能力

BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generation

  • 将语言推理与视觉预测交织融入动作生成流程
  • 在多阶段任务中成功率显著优于现有方法
  • 适合需要长期规划的机器人操作场景

赋予具身智能体任务推理、物理结果预判和精确动作生成的能力,是实现通用操作的关键。尽管近期视觉-语言-动作(VLA)模型利用预训练基础模型取得了进展,但通常仅关注语言规划或视觉预测之一,极少同时整合两者以指导动作生成,导致在复杂长周期操作任务中表现不佳。为此,我们提出BagelVLA,一个统一框架,将语言规划、视觉预测和动作生成集成于单一模型中。该模型基于预训练的统一理解与生成模型初始化,并训练为在动作执行循环中交错进行文本推理与视觉预测。为高效耦合多模态信息,引入残差流引导(RFG),从当前观测出发,利用单步去噪提取预测性视觉特征,以极低延迟引导动作生成。大量实验表明,BagelVLA在多个模拟与真实世界基准上显著超越现有基线,尤其在需多阶段推理的任务中表现突出。

原文摘要 · Abstract (English)

Equipping embodied agents with the ability to reason about tasks, foresee physical outcomes, and generate precise actions is essential for general-purpose manipulation. While recent Vision-Language-Action (VLA) models have leveraged pre-trained foundation models, they typically focus on either linguistic planning or visual forecasting in isolation. These methods rarely integrate both capabilities simultaneously to guide action generation, leading to suboptimal performance in complex, long-horizon manipulation tasks. To bridge this gap, we propose BagelVLA, a unified model that integrates linguistic planning, visual forecasting, and action generation within a single framework. Initialized from a pretrained unified understanding and generative model, BagelVLA is trained to interleave textual reasoning and visual prediction directly into the action execution loop. To efficiently couple these modalities, we introduce Residual Flow Guidance (RFG), which initializes from current observation and leverages single-step denoising to extract predictive visual features, guiding action generation with minimal latency. Extensive experiments demonstrate that BagelVLA outperforms existing baselines by a significant margin on multiple simulated and real-world benchmarks, particularly in tasks requiring multi-stage reasoning.

机器人操作视觉语言长时序生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。