让强化学习模型通过参数化奖励快速适应新目标，无需重新训练。
Reward-Conditioned Reinforcement Learning
- 用共享经验数据重计算不同奖励，实现一次训练多目标适应。
- 在标准任务上提升采样效率，新奖励下可快速适应。
- 适合需要灵活调整行为的部署场景，如机器人控制、个性化推荐。
单任务强化学习通常在固定奖励函数下训练，限制了对奖励错误设定的鲁棒性及对偏好变化的适应能力。我们提出奖励条件强化学习（RCRL），一种离线策略方法，使智能体在单一名义目标下收集经验的同时，对奖励参数化进行条件约束。通过从共享回放数据中重新计算反事实奖励，RCRL 在不增加环境交互的前提下，使智能体暴露于多种奖励目标，将单任务RL与多目标、多任务学习思想相连接。在单任务、多任务及基于视觉的基准测试中，RCRL在名义奖励参数化下提升样本效率，实现对新参数化的高效适应，并支持部署时零样本行为调整。结果表明，RCRL提供了一种无需牺牲单任务训练简单性的可扩展鲁棒、可调控策略学习机制。
原文摘要 · Abstract (English)
Single-task RL agents are typically trained under a fixed reward function, which limits their robustness to reward misspecification and their ability to adapt to changing preferences. We introduce Reward-Conditioned Reinforcement Learning (RCRL), an off-policy method that conditions agents on reward parameterizations while collecting experience under a single nominal objective. By recomputing counterfactual rewards from shared replay data, RCRL exposes the agent to multiple reward objectives without additional environment interaction, connecting single-task RL with ideas from multi-objective and multi-task learning. Across single-task, multi-task, and vision-based benchmarks, RCRL improves sample efficiency under the nominal reward parameterization, enables efficient adaptation to new parameterizations, and supports zero-shot behavioral adjustment at deployment. Our results show that RCRL provides a scalable mechanism for learning robust, steerable policies without sacrificing the simplicity of single-task training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。