用验证器提升生成式机器人策略的鲁棒性,无需重新训练。
EVE: A Generator-Verifier System for Generative Policies
- 引入生成-验证框架,用零样本视觉语言模型修正动作
- 在多种仿真与真实机器人任务中成功率显著提升
- 适合希望不训练就增强策略性能的研究者
基于生成模型(如扩散模型和流匹配)的视觉运动策略在机器人应用中表现强劲,但在分布外场景下性能下降,缺乏有效恢复能力,且需昂贵微调。在语言建模领域,推理时计算扩展通过候选解精炼极大提升了大模型的推理能力,通常利用基础模型作为零样本验证模块进行解评分。我们假设生成策略同样可通过额外推理时计算,借助零样本视觉语言模型验证器,在生成-验证框架中获益。为此,我们提出EVE:一种模块化、无需训练的生成-验证交互框架,可提升预训练生成策略在测试时的表现。EVE将冻结的基线策略封装于多个零样本、基于视觉语言模型的验证器代理中,每个验证器对基线策略的候选动作提出修正建议,而动作融合模块使用分类器引导将聚合的验证反馈融入动作去噪过程。我们在多种模拟与真实机器人任务及形态上研究了生成-验证信息交互的设计选择。实验表明,EVE在不增加策略或验证器训练的前提下,持续提升成功率。通过大量消融实验,我们分离出验证器能力与动作融合策略的贡献,为构建可扩展、模块化的具身控制生成-验证系统提供了实用指导。
原文摘要 · Abstract (English)
Visuomotor policies based on generative such as diffusion and flow-matching have shown strong performance for robotics applications but degrade under distribution shifts, demonstrating limited recovery capabilities without costly finetuning. In the language modeling domain, test-time compute scaling has revolutionized the reasoning capabilities of modern LLMs by enabling candidate solution refinement. These methods typically leverage foundation models as verification modules in a zero-shot manner to score candidate solutions. We hypothesize that generative policies can similarly benefit from additional inference-time compute that employs zero-shot VLM-based verifiers in a generation-verification framework. To this end, we introduce EVE: a modular, generator-verifier interaction framework that boosts the performance of pretrained generative policies at test time, with no additional training. EVE wraps a frozen base policy with multiple zero-shot, VLM-based verifier agents. Each verifier proposes action refinements to the base policy candidate actions, while an action incorporator uses classifier guidance to fuse aggregated verifier feedback into action denoising. We study design choices for generator-verifier information interfacing across a system of verifiers with distinct capabilities. Across diverse simulated and real robotic tasks and embodiments, EVE consistently improves success rates without additional policy or verifier training. Through extensive ablations, we isolate the contribution of verifier capabilities and action incorporator strategies, offering practical guidelines to build scalable, modular generator-verifier systems for embodied control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。