让强化学习模型在未知视觉干扰下仍能稳定表现
Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions
- 用双对比学习提取多视角下的任务相关特征
- 在复杂自然背景上性能比之前方法提升117%
- 适合需要强泛化能力的机器人控制场景
基于模型的强化学习（MBRL）在高维图像观测的视觉控制任务中表现出高效性。然而，现有算法在训练数据之外出现视觉干扰（如云、阴影、光线）时表现下降。本文提出自监督方法 Dream to Generalize（Dr. G），通过双对比学习在多视角数据增强下训练编码器与世界模型，有效捕捉任务相关特征，并引入循环状态逆动力学模型以更好理解时间结构。在 DeepMind Control 套件中于简单背景训练后，测试于复杂自然视频背景；在 Robosuite 随机环境中的测试显示，相比先前方法分别提升 117% 和 14% 的性能。代码已开源。
原文摘要 · Abstract (English)
Model-based reinforcement learning (MBRL) has been used to efficiently solve vision-based control tasks in highdimensional image observations. Although recent MBRL algorithms perform well in trained observations, they fail when faced with visual distractions in observations. These task-irrelevant distractions (e.g., clouds, shadows, and light) may be constantly present in real-world scenarios. In this study, we propose a novel self-supervised method, Dream to Generalize (Dr. G), for zero-shot MBRL. Dr. G trains its encoder and world model with dual contrastive learning which efficiently captures task-relevant features among multi-view data augmentations. We also introduce a recurrent state inverse dynamics model that helps the world model to better understand the temporal structure. The proposed methods can enhance the robustness of the world model against visual distractions. To evaluate the generalization performance, we first train Dr. G on simple backgrounds and then test it on complex natural video backgrounds in the DeepMind Control suite, and the randomizing environments in Robosuite. Dr. G yields a performance improvement of 117% and 14% over prior works, respectively. Our code is open-sourced and available at https://github.com/JeongsooHa/DrG.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。