arXiv:2501.17311cs.ROcs.LG2025-01ICRA被引 6

用强化学习优化纯追逐控制器,实现零样本真实赛车自动控制。

RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms

  • 以纯追逐为基底,用强化学习做残差修正,结合可靠性与自适应性。
  • 在F1TENTH平台实测,最快圈速提升6.37%,逼近顶尖方法性能的52%以上。
  • 无需仿真到现实迁移训练,开箱即用,适合追求实用部署的自动驾驶研究者。

自主赛车需在动态环境中快速决策,传统基于轮胎模型的控制器虽可靠但需大量调参。强化学习能直接从交互中学习,但普遍面临仿真到现实的差距问题。本文提出RLPP,一种残差强化学习框架,将强化学习作为纯追逐(PP)控制器的残差部分进行增强。该混合方法既保留了PP的可解释性与稳定性,又通过强化学习在真实场景中微调表现。在F1TENTH平台上广泛测试表明,RLPP使基线控制器的圈速最快提升6.37%,缩小与最先进方法的差距超过52%,并实现真正的零样本现实部署。相比基线强化学习控制器,其仿真到现实性能差距缩小了8倍以上。代码已开源,网址:www.github.com/forzaeth/rlpp。

原文摘要 · Abstract (English)

Autonomous racing presents a complex environment requiring robust controllers capable of making rapid decisions under dynamic conditions. While traditional controllers based on tire models are reliable, they often demand extensive tuning or system identification. Reinforcement Learning (RL) methods offer significant potential due to their ability to learn directly from interaction, yet they typically suffer from the sim-to-real gap, where policies trained in simulation fail to perform effectively in the real world. In this paper, we propose RLPP, a residual RL framework that enhances a Pure Pursuit (PP) controller with an RL-based residual. This hybrid approach leverages the reliability and interpretability of PP while using RL to fine-tune the controller's performance in real-world scenarios. Extensive testing on the F1TENTH platform demonstrates that RLPP improves lap times of the baseline controllers by up to 6.37 %, closing the gap to the State-of-the-Art methods by more than 52 % and providing reliable performance in zero-shot real-world deployment, overcoming key challenges associated with the sim-to-real transfer and reducing the performance gap from simulation to reality by more than 8-fold when compared to the baseline RL controller. The RLPP framework is made available as an open-source tool, encouraging further exploration and advancement in autonomous racing research. The code is available at: www.github.com/forzaeth/rlpp.

强化学习自动驾驶零样本赛车控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。