改进经验回放机制，让智能体更高效学习过去经验。
Reliability-Adjusted Prioritized Experience Replay
- 根据时序差分误差的可靠性动态调整经验采样优先级
- 在Atari-10等环境中性能优于传统优先经验回放
- 适合追求数据效率的在线强化学习系统
经验回放使在线强化学习代理能够高效利用过往经验。传统方法对经验缓冲区中的样本进行均匀采样，忽略不同经验的学习潜力差异。为提升采样效率，研究者提出了优先经验回放（PER）。本文提出PER的扩展方法——可靠性调整的优先经验回放（ReaPER），引入一种新的时序差分误差可靠性度量。理论证明，ReaPER相比PER能实现更高效的训练。实验结果表明，ReaPER在多种环境（包括Atari-10基准）中均优于PER。
原文摘要 · Abstract (English)
Experience replay enables data-efficient learning from past experiences in online reinforcement learning agents. Traditionally, experiences were sampled uniformly from a replay buffer, regardless of differences in experience-specific learning potential. In an effort to sample more efficiently, researchers introduced Prioritized Experience Replay (PER). In this paper, we propose an extension to PER by introducing a novel measure of temporal difference error reliability. We theoretically show that the resulting transition selection algorithm, Reliability-adjusted Prioritized Experience Replay (ReaPER), enables more efficient learning than PER. We further present empirical results showing that ReaPER outperforms PER across various environment types, including the Atari-10 benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。