用CLIP思想对齐语言与策略表示，让AI更快学会新任务。
CLIP-RL: Aligning Language and Policy Representations for Task Transfer in Reinforcement Learning
- 借鉴CLIP思想，将任务指令和对应策略映射到统一语义空间。
- 在多个任务上实现更快的迁移速度，提升学习效率。
- 适合需要多任务语言指令控制的强化学习场景。
当前亟需开发能在同一环境中解决多种任务的智能体，尤其当这些任务自然关联语言时。本文提出一种新方法，利用预训练的语言与策略配对构建高效的迁移管道。算法受计算机视觉中对比语言-图像预训练（CLIP）的启发，核心理念是：表达同一概念的不同模态应具有相似表示。此处，任务指令与对应策略代表同一任务概念的两种模态。通过将CLIP思想拓展至强化学习，本方法建立自然语言与策略嵌入的统一表示空间。实验表明，该算法在跨任务迁移中显著提升效率。
原文摘要 · Abstract (English)
Recently, there has been an increasing need to develop agents capable of solving multiple tasks within the same environment, especially when these tasks are naturally associated with language. In this work, we propose a novel approach that leverages combinations of pre-trained (language, policy) pairs to establish an efficient transfer pipeline. Our algorithm is inspired by the principles of Contrastive Language-Image Pretraining (CLIP) in Computer Vision, which aligns representations across different modalities under the philosophy that ''two modalities representing the same concept should have similar representations.'' The central idea here is that the instruction and corresponding policy of a task represent the same concept, the task itself, in two different modalities. Therefore, by extending the idea of CLIP to RL, our method creates a unified representation space for natural language and policy embeddings. Experimental results demonstrate the utility of our algorithm in achieving faster transfer across tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。