分离视觉任务相关与无关特征，提升强化学习泛化能力。
Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalization
- 将视觉输入分解为任务相关和无关特征，通过自监督学习实现解耦。
- 在DeepMind控制套件和机器人操作任务中达到当前最优泛化性能。
- 适合关注视觉强化学习泛化与样本效率的研究者。
视觉强化学习（VRL）在解决控制任务上已取得显著进展，但将训练好的策略推广到新环境仍面临挑战，因智能体常过度拟合训练环境中的任务无关特征。为此，本文提出将观测分解为任务相关与任务无关表示的新思路，并设计了自监督的T2RD算法。该算法包含三个组件：任务相关表示一致性、交叉重建和交叉动态预测。前两个组件实现内容与风格特征的解耦，但所得内容表示未必任务相关；第三个组件引入动态预测，进一步筛选出任务相关特征。T2RD在DeepMind Control Suite和机器人操作任务中实现了当前最优的泛化性能与样本效率。
原文摘要 · Abstract (English)
Visual Reinforcement Learning (VRL) has achieved considerable success in solving control tasks. However, generalizing learned policies to new environments remains a major challenge, as agents often overfit to task-irrelevant features in the training environment. To solve this problem, we introduce the concept of decoupling observations into task-relevant and task-irrelevant representations. Building on this idea, we propose a self-supervised Task-Relevant Representation Decoupling (T2RD) algorithm for VRL. This algorithm consists of three components: task-relevant representation consistency, cross-reconstruction, and cross-dynamic prediction. The first two components achieve the decoupling of content and style features, but the resulting content representations are not necessarily task-relevant. To further refine task-relevant features from content representations, we design the third component that introduces dynamic prediction. T2RD achieves State-Of-The-Art (SOTA) generalization performance and sample efficiency in the DeepMind Control Suite and Robotic Manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。