用自适应课程强化学习训练自动驾驶摩托赛车，提升稳定性和速度。
Self-Paced Curriculum Reinforcement Learning for Autonomous Superbike Racing in Simulation
- 通过动态调整任务难度，让智能体逐步掌握复杂骑行技巧。
- 在多条赛道和车型上，比纯SAC方法快12%且更稳定。
- 适合研究自动驾驶两轮车辆或强化学习课程设计的读者。
自主赛车在深度强化学习推动下取得了显著进展，主要集中在四轮车辆。然而，摩托车因需控制平衡与倾斜角度、转向与油门反应更灵敏、重量更轻，带来了更大挑战。本文提出一种在VRider SBK（基于Unity的高精度摩托车模拟器）中训练自动驾驶超级摩托车的方法。该方法将Soft Actor-Critic（SAC）与自适应课程深度强化学习（SPDL）结合，根据智能体表现自动生成渐进式挑战任务，无需人工设计课程。状态空间包含本体感知特征及倾斜角历史，并融合赛道点信息。奖励信号被设计为鼓励沿赛道前进，同时惩罚两轮动力学导致的不稳定性行为。初步实验表明，SPDL在训练效率、圈速和驾驶稳定性方面均优于SAC单一方法，在多个赛道和摩托车模型上建立了基于RL的自主摩托车竞速基准。
原文摘要 · Abstract (English)
Autonomous Racing has seen remarkable progress through deep Reinforcement Learning (RL), primarily for four-wheeled vehicles. However, motorbikes introduce substantially greater complexity due to the need to manage balance and lean angle, in addition to more reactive steering and throttle control, and a smaller weight. In this work, we present a framework for training an autonomous agent to race a superbike in VRider SBK, a physics-accurate Unity-based motorbike simulator. Our approach integrates Soft Actor-Critic (SAC) with Self-Paced curriculum Deep reinforcement Learning (SPDL), which dynamically generates progressively more challenging tasks based on the agent's performance, without requiring manual curriculum design. The agent's state space comprises proprioceptive features extended with lean-angle history, along with global track features via course points. The reward signal is shaped to encourage progress along the track while penalizing instability-inducing behaviors specific to two-wheeled dynamics. Preliminary experimental results demonstrate that SPDL outperforms SAC alone in training efficiency, lap time, and driving stability across multiple tracks and motorbike models, establishing a first baseline for RL-based autonomous motorbike racing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。