首个专为跨动态迁移强化学习设计的基准，评估策略迁移适应能力。
ODRL: A Benchmark for Off-Dynamics Reinforcement Learning
- 构建四类实验场景，覆盖在线/离线源与目标域组合
- 多任务+广谱动态差异，全面测试策略迁移性能
- 提供统一代码框架，支持主流算法对比，适合方法评估者
我们研究离线动态强化学习（off-dynamics RL），即在不同领域间迁移策略时存在动态不匹配的问题。尽管已有动力学感知算法被提出，但该领域因缺乏标准评测基准而受限。为此，我们提出ODRL，首个专为评估离线动态强化学习方法设计的基准。ODRL包含四个实验设置，源域与目标域可为在线或离线，涵盖多样任务与广泛动态偏移，构成一个可靠平台，用于全面评估智能体对目标域的适应能力。此外，ODRL将近期离线动态强化学习算法集成于统一框架，并引入多种设定下的额外基线，均以单文件形式实现。通过大规模基准测试实验，我们发现现有方法在不同动态偏移下无普遍优势。我们希望此基准能成为未来研究的基石。代码公开于 https://github.com/OffDynamicsRL/off-dynamics-rl。
原文摘要 · Abstract (English)
We consider off-dynamics reinforcement learning (RL) where one needs to transfer policies across different domains with dynamics mismatch. Despite the focus on developing dynamics-aware algorithms, this field is hindered due to the lack of a standard benchmark. To bridge this gap, we introduce ODRL, the first benchmark tailored for evaluating off-dynamics RL methods. ODRL contains four experimental settings where the source and target domains can be either online or offline, and provides diverse tasks and a broad spectrum of dynamics shifts, making it a reliable platform to comprehensively evaluate the agent's adaptation ability to the target domain. Furthermore, ODRL includes recent off-dynamics RL algorithms in a unified framework and introduces some extra baselines for different settings, all implemented in a single-file manner. To unpack the true adaptation capability of existing methods, we conduct extensive benchmarking experiments, which show that no method has universal advantages across varied dynamics shifts. We hope this benchmark can serve as a cornerstone for future research endeavors. Our code is publicly available at https://github.com/OffDynamicsRL/off-dynamics-rl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。