arXiv:2602.20659cs.AI2026-02被引 1

提出可长期跟踪任务进展的视觉语言动作模型,解决感知模糊下的执行难题。

Recursive Belief Vision Language Action Models

  • 构建信念机制持续记录任务状态,避免重复查询视觉模型
  • 在多阶段操作任务中成功率提升超50%,推理延迟降低至五分之一
  • 适合需要长时间规划与物理推理的智能体控制场景

视觉-语言-动作模型需在部分可观测环境下执行长时程任务。现有方法多依赖观测驱动,使用短上下文窗口或反复调用视觉语言模型(VLM),导致任务进度丢失、感知混淆下动作重复,且推理延迟高。尽管语义定位重要,但长时程操作核心在于持久、动作相关的状态表征。当前视觉语言动作模型缺乏此类表征,时间与物理推理能力有限,难以胜任多阶段控制。本文提出RB-VLA,一种以信念为中心的架构,通过自监督世界模型目标训练,维护一个紧凑的隐状态,编码任务相关的历史、动态与物体交互。每项任务仅查询一次VLM获取高层意图,信念模块则追踪任务进展,实现无原始观测存储、不随时间增长内存的阶段感知、因果关联控制。信念与意图共同条件化扩散策略,支持鲁棒闭环执行。RB-VLA在长时程基准上优于先前模型,在多阶段抓取放置与堆叠任务中成功率分别比pi_0高出52.5%和37.5%;推理延迟最高降低五倍,且消除现有模型随时间累积内存的问题。消融实验表明信念模块是性能主因,成功率从无信念时的32.5%提升至有信念时的77.5%。

原文摘要 · Abstract (English)

Vision-language-action models must enable agents to execute long-horizon tasks under partial observability. However, most existing approaches remain observation-driven, relying on short context windows or repeated queries to vision-language models (VLMs). This leads to loss of task progress, action repetition under perceptual aliasing, and high inference latency. While semantic grounding is important, long-horizon manipulation fundamentally requires persistent, action-conditioned state representations. Current VLAs lack such representations and exhibit limited temporal and physical reasoning, making them ill-suited for multi-stage control. This paper introduces RB-VLA, a belief-centric architecture trained with self-supervised world-model objectives that maintains a compact latent state encoding task-relevant history, dynamics, and object interactions. Queried once per task, the VLM provides high-level intent, while the belief tracks task progress and enables phase-aware, causally grounded control under partial observability without storing raw observations or scaling memory with time. The belief and intent jointly condition a diffusion policy for robust closed-loop execution. RB-VLA outperforms prior VLAs on long-horizon benchmarks, achieving 52.5 percent and 37.5 percent higher success rates on multi-stage pick-and-place and stacking tasks, respectively, compared to pi_0. It also reduces inference latency by up to five times relative to baselines and eliminates memory growth across timesteps observed in existing VLAs. Ablations show the belief module is the primary driver of performance, increasing success rates from 32.5 percent without belief to 77.5 percent with belief.

视觉语言动作长时程控制信念表示扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。