用遗传算法生成示范数据，提升强化学习在工业场景下的训练效率。
Leveraging Genetic Algorithms for Efficient Demonstration Generation in Real-World Reinforcement Learning Environments
- 用遗传算法生成专家示范轨迹，优化强化学习初始状态。
- 基于遗传算法的示范使PPO模型累计奖励显著提升。
- 适合关注真实世界强化学习落地的研究者与工程师。
强化学习（RL）在部分工业应用中展现出巨大潜力，但其广泛应用仍受限于样本效率低下和学习动态不稳定等问题。本研究探讨了将遗传算法（GAs）用于改进工业级分拣环境中的强化学习性能。提出一种新方法：利用遗传算法生成专家示范数据，将其注入深度Q网络（DQN）的经验回放缓冲区以支持基于经验的学习，并作为近端策略优化（PPO）代理的预热轨迹，加速训练收敛。实验对比了标准强化学习、规则启发式、暴力优化及示范数据的表现，结果表明，遗传算法生成的示范数据能显著提升强化学习性能。值得注意的是，以遗传算法生成数据初始化的PPO代理获得了更优的累积奖励，凸显了启发式搜索与数据驱动强化学习融合的潜力。所提出的框架已公开，便于进一步研究真实世界应用中的自适应强化学习策略。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) has demonstrated significant potential in certain real-world industrial applications, yet its broader deployment remains limited by inherent challenges such as sample inefficiency and unstable learning dynamics. This study investigates the utilization of Genetic Algorithms (GAs) as a mechanism for improving RL performance in an industrially inspired sorting environment. We propose a novel approach in which GA-generated expert demonstrations are used to enhance policy learning. These demonstrations are incorporated into a Deep Q-Network (DQN) replay buffer for experience-based learning and utilized as warm-start trajectories for Proximal Policy Optimization (PPO) agents to accelerate training convergence. Our experiments compare standard RL training with rule-based heuristics, brute-force optimization, and demonstration data, revealing that GA-derived demonstrations significantly improve RL performance. Notably, PPO agents initialized with GA-generated data achieved superior cumulative rewards, highlighting the potential of hybrid learning paradigms, where heuristic search methods complement data-driven RL. The utilized framework is publicly available and enables further research into adaptive RL strategies for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。