arXiv:2604.22238cs.RO2026-04被引 1

用代码规划+语义图状态,让机器人长程操作更可靠

CodeGraphVLP: Code-as-Planner Meets Semantic-Graph State for Non-Markovian Vision-Language-Action Models

论文配图:CodeGraphVLP: Code-as-Planner Meets Semantic-Graph State for Non-Markovian Vision-Language-Action Models
图 1 · 摘自论文原文
  • 构建持久语义图记录关键实体关系,应对部分可观测场景
  • 代码规划器生成子任务指令,减少视觉干扰并提升执行效率
  • 适合复杂现实场景中需长期推理的机器人操作任务

视觉-语言-动作(VLA)模型有望实现通用机器人操作,但通常作为短时序策略训练与部署,假设最新观测足以支持动作推理。这一假设在非马尔可夫的长时序任务中失效,因任务相关证据可能被遮挡或早期出现,且杂乱环境使细粒度视觉定位变得脆弱。本文提出CodeGraphVLP,一种分层框架,通过结合持久的语义图状态、可执行代码规划器及进度引导的视觉-语言提示,实现可靠的长时序操作。语义图在部分可观测条件下持续维护任务相关实体及其关系;合成规划器基于该图进行高效进展检查,并输出子任务指令及关联物体。利用这些输出构建抑制杂乱的观测,聚焦于关键证据。在真实世界非马尔可夫任务上,CodeGraphVLP优于强基线和历史增强变体,同时显著降低规划延迟,相比端到端视觉语言模型规划。我们还进行了大量消融实验,验证各组件贡献。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models promise generalist robot manipulation, but are typically trained and deployed as short-horizon policies that assume the latest observation is sufficient for action reasoning. This assumption breaks in non-Markovian long-horizon tasks, where task-relevant evidence can be occluded or appear only earlier in the trajectory, and where clutter and distractors make fine-grained visual grounding brittle. We present CodeGraphVLP, a hierarchical framework that enables reliable long-horizon manipulation by combining a persistent semantic-graph state with an executable code-based planner and progress-guided visual-language prompting. The semantic-graph maintains task-relevant entities and relations under partial observability. The synthesized planner executes over this semantic-graph to perform efficient progress checks and outputs a subtask instruction together with subtask-relevant objects. We use these outputs to construct clutter-suppressed observations that focus the VLA executor on critical evidence. On real-world non-Markovian tasks, CodeGraphVLP improves task completion over strong VLA baselines and history-enabled variants while substantially lowering planning latency compared to VLM-in-the-loop planning. We also conduct extensive ablation studies to confirm the contributions of each component.

机器人操作长程规划语义图VLA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。