动态熵调优提升强化学习飞行控制稳定性与探索效率
Dynamic Entropy Tuning in Reinforcement Learning Low-Level Quadcopter Control: Stochasticity vs Determinism
- 用动态熵调节替代固定熵，增强策略多样性
- 动态熵使四旋翼控制更稳定，减少灾难性遗忘
- 适合需要高效探索的无人机强化学习研究者
本文研究了在强化学习中使用动态熵调优对随机策略训练的影响，并与确定性策略进行对比。随机策略通过优化动作概率分布来最大化奖励，而确定性策略则在每个状态下选择单一动作。研究采用Soft Actor-Critic（SAC）作为随机策略算法，Twin Delayed Deep Deterministic Policy Gradient（TD3）作为确定性策略算法。实验结果表明，动态熵调优能有效防止灾难性遗忘，提升探索效率，显著改善四旋翼飞行器的控制性能。
原文摘要 · Abstract (English)
This paper explores the impact of dynamic entropy tuning in Reinforcement Learning (RL) algorithms that train a stochastic policy. Its performance is compared against algorithms that train a deterministic one. Stochastic policies optimize a probability distribution over actions to maximize rewards, while deterministic policies select a single deterministic action per state. The effect of training a stochastic policy with both static entropy and dynamic entropy and then executing deterministic actions to control the quadcopter is explored. It is then compared against training a deterministic policy and executing deterministic actions. For the purpose of this research, the Soft Actor-Critic (SAC) algorithm was chosen for the stochastic algorithm while the Twin Delayed Deep Deterministic Policy Gradient (TD3) was chosen for the deterministic algorithm. The training and simulation results show the positive effect the dynamic entropy tuning has on controlling the quadcopter by preventing catastrophic forgetting and improving exploration efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。