让人类实时指导自动驾驶强化学习，提升安全与适应性。
Interactive Double Deep Q-network: Integrating Human Interventions and Evaluative Predictions in Reinforcement Learning of Autonomous Driving
- 将人类干预直接融入强化学习的Q值更新，实现人机协同决策
- 在模拟场景中表现优于行为克隆、DQfD等基准方法
- 提供离线评估框架，量化人类干预的实际效果
将人类专长与机器学习结合对高精度高安全性应用（如自动驾驶）至关重要。本文提出交互式双深度Q网络（iDDQN），一种人机协同强化学习方法，通过修改Q值更新机制，直接融合人类与智能体的动作，实现策略协同优化。此外，设计了一种离线评估框架，模拟无干预情形下的智能体轨迹，以评估人类干预的有效性。在模拟自动驾驶场景中的实验证明，iDDQN在利用人类经验提升性能与适应性方面，显著优于行为克隆（BC）、HG-DAgger、基于示范的深度Q学习（DQfD）及原始强化学习方法。
原文摘要 · Abstract (English)
Integrating human expertise with machine learning is crucial for applications demanding high accuracy and safety, such as autonomous driving. This study introduces Interactive Double Deep Q-network (iDDQN), a Human-in-the-Loop (HITL) approach that enhances Reinforcement Learning (RL) by merging human insights directly into the RL training process, improving model performance. Our proposed iDDQN method modifies the Q-value update equation to integrate human and agent actions, establishing a collaborative approach for policy development. Additionally, we present an offline evaluative framework that simulates the agent's trajectory as if no human intervention had occurred, to assess the effectiveness of human interventions. Empirical results in simulated autonomous driving scenarios demonstrate that iDDQN outperforms established approaches, including Behavioral Cloning (BC), HG-DAgger, Deep Q-Learning from Demonstrations (DQfD), and vanilla DRL in leveraging human expertise for improving performance and adaptability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。