提出自预测动态方法，提升视觉强化学习在复杂干扰下的泛化能力
Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning
- 通过双向增强图像的逆向与正向预测，学习任务相关特征
- 在MuJoCo和CARLA任务中显著提升未见观测下的泛化性能
- 适合处理训练时未暴露的视觉干扰，如阴影、云层等
基于视觉的强化学习需要高效且鲁棒的图像观测表征，尤其当图像包含阴影、云层、光线等干扰元素（任务无关）时更为关键。若这些干扰在训练中未被暴露，问题将更严峻。本文设计了自预测动态（SPD）方法，在训练后面对未见观测时仍能高效提取任务相关特征。SPD并行使用弱增强与强增强图像，通过预测两组增强版本之间的逆向与正向转换关系来学习表征。在一系列MuJoCo视觉控制任务及自动驾驶任务（CARLA）中，SPD在复杂观测条件下优于以往方法，显著提升了对未见观测的泛化性能。代码已开源：https://github.com/unigary/SPD。
原文摘要 · Abstract (English)
Vision-based reinforcement learning requires efficient and robust representations of image-based observations, especially when the images contain distracting (task-irrelevant) elements such as shadows, clouds, and light. It becomes more important if those distractions are not exposed during training. We design a Self-Predictive Dynamics (SPD) method to extract task-relevant features efficiently, even in unseen observations after training. SPD uses weak and strong augmentations in parallel, and learns representations by predicting inverse and forward transitions across the two-way augmented versions. In a set of MuJoCo visual control tasks and an autonomous driving task (CARLA), SPD outperforms previous studies in complex observations, and significantly improves the generalization performance for unseen observations. Our code is available at https://github.com/unigary/SPD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。