提出统一框架，让智能体零样本适配任意奖励函数。
A Unified Framework for Zero-Shot Reinforcement Learning
- 按表示方式分组合与直接两类，按训练范式分无奖励与伪无奖励两类。
- 将总误差分解为推理、奖励、近似三部分，可量化比较不同方法。
- 适合研究零样本强化学习的算法设计与理论分析者阅读。
零样本强化学习（Zero-shot RL）旨在构建无需额外训练或规划即可解决下游任务的通用智能体。传统RL针对固定奖励优化策略，而零样本RL需学习能立即适应任意奖励函数的表征。随着方法多样化，亟需一个基础框架来统一不同视角。本文提出形式化的统一框架，支持方法间严格比较。我们构建了两层分类体系：表示层面区分组合式与直接式方法，基于对动作-价值函数分解的利用；学习范式层面区分无奖励与伪无奖励训练。此外，我们提出统一的误差界观点，将总误差分解为推理误差、奖励误差和近似误差三部分，为零样本方法的更扎实比较提供基础。
原文摘要 · Abstract (English)
Zero-shot reinforcement learning (RL) has emerged as a setting for developing general agents, capable of solving downstream tasks without additional training or planning at test-time. While conventional RL optimizes policies for fixed rewards, zero-shot RL requires learning representations that enable immediate adaptation to arbitrary reward functions. As the field matures, the growing diversity of approaches demands a foundational framework reconciling different perspectives under a common unifying structure. In this work, we introduce a formal, unified framework for zero-shot RL, allowing for rigorous comparisons across methods. We propose a taxonomy organizing the algorithmic landscape along two levels: representation, distinguishing between compositional and direct methods based on their exploitation of action-value function decompositions; and learning paradigm, differentiating between reward-free and pseudo reward-free training. Additionally, we propose a unified view of existing error bounds, decomposing the total error into three primary contributing components: inference, reward, and approximation, serving as a foundation for more grounded comparisons of zero-shot methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。