通过轨迹解耦实现模拟到现实的零样本迁移,让自动驾驶强化学习直接上车。
Dynamics-Decoupled Trajectory Alignment for Sim-to-Real Transfer in Reinforcement Learning for Autonomous Driving
- 将轨迹生成与车辆控制分离,用虚拟与实车对齐策略降低动力学差异影响。
- 在真实车辆上实现零样本迁移,无需微调即可稳定执行模拟训练的规划策略。
- 适合做自动驾驶仿真-现实迁移的研究者或工程团队参考。
强化学习(RL)在机器人领域展现出巨大潜力,但将其部署于真实车辆仍面临挑战,主要源于车辆动力学复杂性及仿真与现实之间的差距。轮胎特性、路面状况、空气动力干扰和载荷变化等因素使得真实动力学难以精确建模,阻碍了仿真中训练的RL智能体直接迁移到现实。本文提出一种框架,通过时空对齐策略将运动规划与车辆控制解耦。首先在仿真中使用运动学自行车模型训练一个RL智能体,输出连续控制动作;其行为被提炼为轨迹预测智能体,生成有限时域的本车轨迹,实现虚拟与真实车辆的同步。部署时,斯坦利控制器负责横向动力学,纵向则通过自适应更新机制补偿虚拟与真实轨迹间的偏差。我们在真实车辆上验证了该方法,结果表明该对齐策略可实现基于强化学习的运动规划从仿真到现实的鲁棒零样本迁移,成功将高层轨迹生成与底层车辆控制解耦。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has shown promise in robotics, but deploying RL on real vehicles remains challenging due to the complexity of vehicle dynamics and the mismatch between simulation and reality. Factors such as tire characteristics, road surface conditions, aerodynamic disturbances, and vehicle load make it infeasible to model real-world dynamics accurately, which hinders direct transfer of RL agents trained in simulation. In this paper, we present a framework that decouples motion planning from vehicle control through a spatial and temporal alignment strategy between a virtual vehicle and the real system. An RL agent is first trained in simulation using a kinematic bicycle model to output continuous control actions. Its behavior is then distilled into a trajectory-predicting agent that generates finite-horizon ego-vehicle trajectories, enabling synchronization between virtual and real vehicles. At deployment, a Stanley controller governs lateral dynamics, while longitudinal alignment is maintained through adaptive update mechanisms that compensate for deviations between virtual and real trajectories. We validate our approach on a real vehicle and demonstrate that the proposed alignment strategy enables robust zero-shot transfer of RL-based motion planning from simulation to reality, successfully decoupling high-level trajectory generation from low-level vehicle control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。