通过不确定性估计优化探索，提升强化学习训练效率。
DreamerV3-XP: Optimizing exploration through uncertainty estimation
- 用返回值、重建误差和价值误差加权轨迹，实现优先经验回放。
- 在稀疏奖励任务中，学习速度更快，动态模型损失更低。
- 适合研究高效探索与世界模型的学者参考。
我们提出 DreamerV3-XP，是 DreamerV3 的改进版本，旨在提升探索能力和学习效率。该方法包含两点：(i) 基于回报、重构损失和价值误差对轨迹进行评分，构建优先经验回放缓冲区；(ii) 利用一组世界模型对未来环境奖励的预测分歧，生成内在奖励信号。在 Atari100k 子集和 DeepMind Control Visual Benchmark 任务上的评估表明，DreamerV3-XP 保持了原始 DreamerV3 的性能，并在稀疏奖励设置下实现了更快的学习速度和更低的动力学模型损失。
原文摘要 · Abstract (English)
We introduce DreamerV3-XP, an extension of DreamerV3 that improves exploration and learning efficiency. This includes (i) a prioritized replay buffer, scoring trajectories by return, reconstruction loss, and value error and (ii) an intrinsic reward based on disagreement over predicted environment rewards from an ensemble of world models. DreamerV3-XP is evaluated on a subset of Atari100k and DeepMind Control Visual Benchmark tasks, confirming the original DreamerV3 results and showing that our extensions lead to faster learning and lower dynamics model loss, particularly in sparse-reward settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。