混合DQN-TD3实现动态环境自主导航，提升成功率与平滑性
Hybrid DQN-TD3 Reinforcement Learning for Autonomous Navigation in Dynamic Environments
- 高层用DQN选子目标，底层用TD3生成连续控制指令
- 在动态环境中成功率达92.3%，碰撞率低于5.1%，重规划效率提升40%
- 适合做机器人路径规划、强化学习部署的开发者参考
本文提出一种分层式路径规划与控制框架，结合高层深度Q网络（DQN）进行离散子目标选择和低层双延迟深度确定性策略梯度（TD3）控制器执行连续动作。高层模块负责行为与子目标决策，低层模块输出平滑速度指令。设计了包含方向、距离、避障、动作平滑性、碰撞惩罚、时间惩罚和进展奖励的综合奖励函数，并引入基于激光雷达的安全门机制以防止危险动作。系统基于ROS + Gazebo（TurtleBot3）实现，在动态且部分可观测环境中使用PathBench评估指标（成功率、碰撞率、路径效率、重规划效率）进行测试。实验表明，该方法相比单一算法基线（DQN或TD3）及规则式规划器，在成功率、样本效率和未见障碍配置下的泛化能力方面均有提升，同时显著减少控制突变。代码与评估脚本已在项目仓库公开。
原文摘要 · Abstract (English)
This paper presents a hierarchical path-planning and control framework that combines a high-level Deep Q-Network (DQN) for discrete sub-goal selection with a low-level Twin Delayed Deep Deterministic Policy Gradient (TD3) controller for continuous actuation. The high-level module selects behaviors and sub-goals; the low-level module executes smooth velocity commands. We design a practical reward shaping scheme (direction, distance, obstacle avoidance, action smoothness, collision penalty, time penalty, and progress), together with a LiDAR-based safety gate that prevents unsafe motions. The system is implemented in ROS + Gazebo (TurtleBot3) and evaluated with PathBench metrics, including success rate, collision rate, path efficiency, and re-planning efficiency, in dynamic and partially observable environments. Experiments show improved success rate and sample efficiency over single-algorithm baselines (DQN or TD3 alone) and rule-based planners, with better generalization to unseen obstacle configurations and reduced abrupt control changes. Code and evaluation scripts are available at the project repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。