通过动态和相对动作空间缩减,提升自动驾驶强化学习训练效率。
Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving
- 引入动态掩码与相对缩减策略,根据上下文实时过滤无效动作。
- 实验显示该方法使训练更稳定,控制精度与泛化能力显著提升。
- 适合追求高效可靠自动驾驶强化学习的开发者与研究者。
强化学习(RL)为自动驾驶提供了通过环境交互学习控制策略的前景,但高维且庞大的动作空间常导致训练效率低、探索成本高。本研究提出并评估两种新型结构化动作空间优化策略:动态掩码与相对动作空间缩减。这些方法与固定缩减方案及完整动作空间基线进行系统对比,以评估其对策略学习与性能的影响。框架采用多模态近端策略优化代理,处理语义图像序列与车辆状态数值。所提出的动态与相对策略基于实时上下文与状态转移进行动作掩码,保持动作一致性的同时剔除无效或次优选择。在多种驾驶路线的综合实验中,动作空间缩减显著提升了训练稳定性与策略性能。动态与相对方案尤其在学习速度、控制精度与泛化能力间取得良好平衡。结果表明,上下文感知的动作空间设计对可扩展、可靠的自动驾驶强化学习至关重要。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) offers a promising framework for autonomous driving by enabling agents to learn control policies through interaction with environments. However, large and high-dimensional action spaces often used to support fine-grained control can impede training efficiency and increase exploration costs. In this study, we introduce and evaluate two novel structured action space modification strategies for RL in autonomous driving: dynamic masking and relative action space reduction. These approaches are systematically compared against fixed reduction schemes and full action space baselines to assess their impact on policy learning and performance. Our framework leverages a multimodal Proximal Policy Optimization agent that processes both semantic image sequences and scalar vehicle states. The proposed dynamic and relative strategies incorporate real-time action masking based on context and state transitions, preserving action consistency while eliminating invalid or suboptimal choices. Through comprehensive experiments across diverse driving routes, we show that action space reduction significantly improves training stability and policy performance. The dynamic and relative schemes, in particular, achieve a favorable balance between learning speed, control precision, and generalization. These findings highlight the importance of context-aware action space design for scalable and reliable RL in autonomous driving tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。