对比强化学习生成测试场景,发现多目标方法更全面,单目标方法更易暴露严重问题。
Reinforcement Learning for Testing Interdependent Requirements in Autonomous Vehicles: An Empirical Study
- 用多目标与单目标强化学习生成自动驾驶测试场景
- 多目标方法覆盖更多场景,单目标方法产生更严重违规
- 适用于关注测试多样性或严重性风险的开发者
自动驾驶车辆决策无需人工干预,保障其可靠性至关重要。基于场景的测试广泛用于评估车辆在多种条件下的表现,强化学习(RL)可生成测试场景以识别功能与安全要求的违反。许多需求相互依赖且存在权衡,因此难以判断单目标强化学习(SORL)是否能可靠揭示违规,还是必须采用多目标强化学习(MORL)来显式处理多个目标。本文通过端到端自动驾驶控制器与高保真模拟器,对SORL与MORL在生成同时测试互依需求的极端场景方面进行了实证比较。结果表明,两者主要差异在于违规发生方式,多数情况下效果相当:MORL生成更多违规场景,而SORL引发更高严重性违规。相对性能还取决于具体目标组合,以及一定程度上的道路条件。在多样性方面,MORL始终覆盖更广场景。因此,若重视场景覆盖率,推荐使用MORL;若关注严重违规,则SORL可能更优。本研究系统比较了两种方法,填补了该领域空白,强调了需求依赖性在基于强化学习的自动驾驶测试中的重要性。
原文摘要 · Abstract (English)
Autonomous vehicles (AVs) make driving decisions without humans, making dependability assurance critical. Scenario-based testing is widely used to evaluate AVs under diverse conditions, with reinforcement learning (RL) generating test scenarios that identify violations of functional and safety requirements. Many requirements are interdependent and involve trade-offs, making it unclear whether single-objective RL (SORL), which combines objectives into a single reward, can reliably reveal violations or whether multi-objective RL (MORL), which explicitly considers multiple objectives, is necessary. We present an empirical evaluation comparing SORL and MORL for generating critical scenarios that simultaneously test interdependent requirements using an end-to-end AV controller and high-fidelity simulator. Results suggest that MORL and SORL differ mainly in how violations occur, while showing comparable effectiveness in many cases. MORL tends to generate more requirement-violation scenarios, whereas SORL produces higher-severity violations. Their relative performance also depends on specific objective combinations and, to a lesser extent, road conditions. Regarding diversity, MORL consistently covers a broader range of scenarios. Thus, MORL is preferable when scenario diversity and coverage are prioritized, whereas SORL may better expose severe violations. Our empirical evaluation addresses a gap by systematically comparing SORL and MORL, highlighting the importance of requirement dependencies in RL-based AV testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。