提出闭环视觉-语言-动作模型,提升机器人在干扰下的操作鲁棒性。
GEVRM: Goal-Expressive Video Generation Model For Robust Visual Manipulation
- 基于内模控制原理,构建闭环系统生成可表达的目标视频。
- 在扰动环境下实现比现有方法高18.3%的基准表现提升。
- 适合需要抗干扰能力的现实机器人任务场景。
随着具身人工智能的快速发展,视觉-语言-动作(VLA)模型在通用机器人决策方面取得了显著进展。然而,大多数现有VLA在部署过程中无法应对不可避免的外部扰动,这些扰动会引入未预期的状态信息,导致动作不准确,从而显著降低泛化性能。经典内模控制(IMC)原理表明,包含外部输入信号的闭环系统能够准确跟踪参考输入并有效抵消干扰。本文提出新型闭环VLA方法GEVRM,融合IMC原理以增强机器人视觉操作的鲁棒性。GEVRM中的文本引导视频生成模型可生成高度表达性的未来视觉规划目标。同时,通过模拟响应评估扰动,生成称为内部嵌入的特征,并通过原型对比学习进行优化,使模型能隐式推断并区分外部环境扰动。所提GEVRM在标准和受扰的CALVIN基准上均达到当前最佳性能,并在真实机器人任务中表现出显著改进。
原文摘要 · Abstract (English)
With the rapid development of embodied artificial intelligence, significant progress has been made in vision-language-action (VLA) models for general robot decision-making. However, the majority of existing VLAs fail to account for the inevitable external perturbations encountered during deployment. These perturbations introduce unforeseen state information to the VLA, resulting in inaccurate actions and consequently, a significant decline in generalization performance. The classic internal model control (IMC) principle demonstrates that a closed-loop system with an internal model that includes external input signals can accurately track the reference input and effectively offset the disturbance. We propose a novel closed-loop VLA method GEVRM that integrates the IMC principle to enhance the robustness of robot visual manipulation. The text-guided video generation model in GEVRM can generate highly expressive future visual planning goals. Simultaneously, we evaluate perturbations by simulating responses, which are called internal embeddings and optimized through prototype contrastive learning. This allows the model to implicitly infer and distinguish perturbations from the external environment. The proposed GEVRM achieves state-of-the-art performance on both standard and perturbed CALVIN benchmarks and shows significant improvements in realistic robot tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。