arXiv:2604.12628math.OCcs.RO2026-04被引 1

用强化学习加速自动驾驶路径规划,实时性更强但有局限。

A Comparison of Reinforcement Learning and Optimal Control Methods for Path Planning

  • 用DDPG强化学习直接映射状态到动作,实现快速决策
  • 仿真中速度远超传统最优控制方法,适合实时应用
  • 能识别起点可行区域,助力任务预规划

在充满威胁的环境中,自主车辆路径规划是基础挑战。传统最优控制方法虽能生成理想路径,但计算耗时过长,难以满足实时决策需求。为此,本文提出基于深度确定性策略梯度(DDPG)的方法,将威胁建模为简单的圆形“禁区”。若车辆进入该禁区或未抵达目标邻域,则判定任务失败。通过奖励函数与两个神经网络(演员-评论家)描述环境并指导控制。训练后的DDPG代理可学习从当前状态(位置与速度)到一系列可行动作的映射,以安全抵达目标。该方法能有效识别出所有可安全抵达目标的起始点集合(“可行集”),为任务规划提供关键信息。在仿真中,与传统的伪谱最优控制方法对比显示,学习型代理可生成有效路径且显著更快,更适合实时应用。然而,部分区域仍属“不可行集”,且可行集内的路径未必最优。这些初步结果指引未来研究:(1) 改进奖励函数以扩大可行集;(2) 分析伪谱方法所得可行集;(3) 探索弧搜索内点法(arc-search IPM)用于路径规划。

原文摘要 · Abstract (English)

Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge. While traditional optimal control methods can find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as a simple, circular `no-go' zone. A mission failure is claimed if the vehicle enters this `no-go' zone at any time or does not reach a neighborhood of the destination. The DDPG agent is trained to learn a direct mapping from its current state (position and velocity) to a series of feasible actions that guide the agent to safely reach its goal. A reward function and two neural networks, critic and actor, are used to describe the environment and guide the control efforts. The DDPG trains the agent to find the largest possible set of starting points (``feasible set'') wherein a safe path to the goal is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based agent may produce effective paths while being significantly faster, making it a better fit for real-time applications. However, there are areas (``infeasible set'') where the DDPG agent cannot find paths to the destination, and the paths in the feasible set may not be optimal. These preliminary results guide our future research: (1) improve the reward function to enlarge the DDPG feasible set, (2) examine the feasible set obtained by the pseudo-spectral method, and (3) investigate the arc-search IPM method for the path planning problem.

强化学习路径规划自动驾驶DDPG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。