用视觉批评者动态调度,让机器人更稳地完成复杂长程操作。
Critic in the Loop: A Tri-System VLA Framework for Robust Long-Horizon Manipulation
- 三系统协同:语言模型负责全局规划,动作模型快速执行,视觉批评者实时监控
- 仅在任务卡顿或失败时调用昂贵的语义模型,减少90%以上无效查询
- 自动打破无限重试循环,适合真实场景中高鲁棒性需求的机器人应用
视觉-语言-动作(VLA)模型在机器人长程操作中面临高阶语义推理与低层实时控制难以兼顾的挑战。为解决此问题,本文提出「批评者在环」框架,采用仿生三系统架构:以视觉语言模型(VLM)作为全局脑,负责高层语义规划;以视觉语言动作(VLA)模型作为小脑,执行快速反应动作;并引入轻量级视觉批评者持续监测工作空间。当检测到任务停滞或失败时,批评者动态切换至VLM进行重规划,否则维持高效闭环执行。该机制显著减少对高延迟语义模型的依赖,避免无限重试,并在多个复杂长程操作基准测试中达到领先性能,尤其在分布外(OOD)场景下表现出更强鲁棒性。
原文摘要 · Abstract (English)
Balancing high-level semantic reasoning with low-level reactive control remains a core challenge in visual robotic manipulation. While Vision-Language Models (VLMs) excel at cognitive planning, their inference latency precludes real-time execution. Conversely, fast Vision-Language-Action (VLA) models often lack the semantic depth required for complex, long-horizon tasks. To bridge this gap, we introduce Critic in the Loop, an adaptive hierarchical framework driven by dynamic VLM-Expert scheduling. At its core is a bionic Tri-System architecture comprising a VLM brain for global reasoning, a VLA cerebellum for reactive execution, and a lightweight visual Critic. By continuously monitoring the workspace, the Critic dynamically routes control authority. It sustains rapid closed-loop execution via the VLA for routine subtasks, and adaptively triggers the VLM for replanning upon detecting execution anomalies such as task stagnation or failures. Furthermore, our architecture seamlessly integrates human-inspired rules to intuitively break infinite retry loops. This visually-grounded scheduling minimizes expensive VLM queries, while substantially enhancing system robustness and autonomy in out-of-distribution (OOD) scenarios. Comprehensive experiments on challenging, long-horizon manipulation benchmarks reveal that our approach achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。