用视觉模型闭环验证,生成更真实可信的长时序操作数据集
V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks
- 动态维护禁入区域地图,避免物体穿插,保证场景几何合理
- 将高阶指令分解为可执行动作,提升长序列规划连贯性
- 通过视觉语言模型逐阶段验证,过滤掉看似运行却无效的任务
从合成数据中学习长时序具身行为仍面临挑战:生成场景常不物理合理,语言程序常在未满足任务语义下“成功”,且高层指令难以转化为可执行动作序列。为此,我们提出V-CAGE,一个闭环框架,用于大规模生成鲁棒且语义对齐的操作数据集。首先,提出上下文感知实例化机制,在场景生成中保持几何一致性。通过动态维护已放置物体的禁入空间区域,系统防止物体相互穿插,确保在杂乱环境中配置可达且无冲突。其次,为弥合抽象意图与底层控制间的鸿沟,引入分层指令分解模块,将高层目标(如“准备上班”)分解为组合式动作基元,支持连贯的长时序规划。关键在于,通过基于视觉语言模型(VLM)的验证循环强制语义正确性。该模型充当视觉批评者,在每个子任务后执行严格拒绝采样,剔除代码执行但未达成视觉目标的“沉默失败”。实验表明,V-CAGE生成的数据集在物理与语义保真度上均显著优于非验证基线,大幅提升了下游策略的成功率与泛化能力。
原文摘要 · Abstract (English)
Learning long-horizon embodied behaviors from synthetic data remains challenging because generated scenes are often physically implausible, language-driven programs frequently "succeed" without satisfying task semantics, and high-level instructions require grounding into executable action sequences. To address these limitations, we introduce V-CAGE, a closed-loop framework for generating robust, semantically aligned manipulation datasets at scale. First, we propose a context-aware instantiation mechanism that enforces geometric consistency during scene synthesis. By dynamically maintaining a map of prohibited spatial areas as objects are placed, our system prevents interpenetration and ensures reachable, conflict-free configurations in cluttered environments. Second, to bridge the gap between abstract intent and low-level control, we employ a hierarchical instruction decomposition module. This decomposes high-level goals (e.g., "get ready for work") into compositional action primitives, facilitating coherent long-horizon planning. Crucially, we enforce semantic correctness through a VLM-based verification loop. Acting as a visual critic, the VLM performs rigorous rejection sampling after each subtask, filtering out "silent failures" where code executes but fails to achieve the visual goal. Experiments demonstrate that V-CAGE yields datasets with superior physical and semantic fidelity, significantly boosting the success rate and generalization of downstream policies compared to non-verified baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。