进化策略可作为深度强化学习预训练起点，但仅在简单任务中有效。
Evolution Strategies for Deep RL pretraining
- 用进化策略做DRL的预训练，无需梯度计算
- 在Flappy Bird中提升训练效率，其他任务无明显改善
- 适合简单环境预训练，复杂任务不推荐使用
尽管深度强化学习在复杂决策问题中表现优异，但其训练需大量计算资源和精细调参。进化策略（ES）提供了一种无需梯度、计算成本更低的替代方法。本文评估了ES与深度强化学习（DRL）在不同难度任务中的表现，包括Flappy Bird、Breakout和MuJoCo环境，并探究了将ES用于初始预训练是否能提升DRL性能。结果表明，ES并未始终比DRL训练更快；作为预训练步骤时，仅在简单环境（Flappy Bird）中带来收益，在复杂任务（Breakout和MuJoCo Walker）中对训练效率或稳定性均无显著提升，且在不同参数设置下效果差异不大。
原文摘要 · Abstract (English)
Although Deep Reinforcement Learning has proven highly effective for complex decision-making problems, it demands significant computational resources and careful parameter adjustment in order to develop successful strategies. Evolution strategies offer a more straightforward, derivative-free approach that is less computationally costly and simpler to deploy. However, ES generally do not match the performance levels achieved by DRL, which calls into question their suitability for more demanding scenarios. This study examines the performance of ES and DRL across tasks of varying difficulty, including Flappy Bird, Breakout and Mujoco environments, as well as whether ES could be used for initial training to enhance DRL algorithms. The results indicate that ES do not consistently train faster than DRL. When used as a preliminary training step, they only provide benefits in less complex environments (Flappy Bird) and show minimal or no improvement in training efficiency or stability across different parameter settings when applied to more sophisticated tasks (Breakout and MuJoCo Walker).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。