arXiv:2411.02559cs.LG2024-11中稿 · ICKG2024被引 7

让DQN动态调整学习重点,快速适应变化环境。

Dynamic Weight Adjusting Deep Q-Networks for Real-Time Environmental Adaptation

  • 根据实时反馈调整经验回放采样概率,聚焦关键过渡
  • 在频繁突变环境中,性能优于传统DQN模型
  • 适合需要快速响应的动态系统控制任务

深度强化学习在复杂任务中表现优异,但常受限于静态训练模式和对稳定环境大量数据的依赖。为解决这一问题,本文将动态权重调整引入深度Q网络(DQN),通过修改经验回放中的采样概率,使模型更关注由实时环境反馈和性能指标指示的关键状态转移。我们提出一种新型交互式动态评估方法(IDEM),基于环境反馈与学习进展优先处理重要转换,显著提升DQN在动态环境中的适应能力。实验表明,在需快速适应的场景下,IDEM-DQN能更有效泛化并稳定学习过程。在多种设置下的对比测试显示,该方法在频繁且不可预测变化的环境中显著优于标准DQN模型。

原文摘要 · Abstract (English)

Deep Reinforcement Learning has shown excellent performance in generating efficient solutions for complex tasks. However, its efficacy is often limited by static training modes and heavy reliance on vast data from stable environments. To address these shortcomings, this study explores integrating dynamic weight adjustments into Deep Q-Networks (DQN) to enhance their adaptability. We implement these adjustments by modifying the sampling probabilities in the experience replay to make the model focus more on pivotal transitions as indicated by real-time environmental feedback and performance metrics. We design a novel Interactive Dynamic Evaluation Method (IDEM) for DQN that successfully navigates dynamic environments by prioritizing significant transitions based on environmental feedback and learning progress. Additionally, when faced with rapid changes in environmental conditions, IDEM-DQN shows improved performance compared to baseline methods. Our results indicate that under circumstances requiring rapid adaptation, IDEM-DQN can more effectively generalize and stabilize learning. Extensive experiments across various settings confirm that IDEM-DQN outperforms standard DQN models, particularly in environments characterized by frequent and unpredictable changes.

强化学习动态适应DQN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。