系统性梳理AI安全中的异常数据检测方法,助力自动驾驶等高风险场景可靠性提升。
Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems
- 从全生命周期视角整合OoD检测技术,支持系统安全论证
- 揭示导致异常数据的根源及安全验证难点
- 适合从事自动驾驶、机器人等安全关键系统研发的工程师
近年来,得益于机器人与机器学习(ML)的发展,人工智能驱动的自主系统在功能和应用范围上均显著扩展。确保这些系统的安全性对于其负责任的应用至关重要,但面临严峻挑战:必须具备能够应对全生命周期中新型和不确定情况的稳健方法,尤其是识别分布外(OoD)数据。因此,OoD检测受到研究、开发和安全工程界的广泛关注。本文系统综述了在自主系统安全保证背景下,特别是安全关键领域中的OoD检测技术。我们首先定义相关概念,探究导致OoD的原因,并分析自主系统安全保证及OoD检测面临的复杂因素。综述识别出一系列可在机器学习开发全生命周期中应用的技术,并建议在哪些环节可使用这些技术以支持安全论证。同时讨论系统与安全工程师在将OoD检测融入系统生命周期时需注意的若干注意事项。最后,提出未来实现跨领域与应用场景下自主系统安全开发与运行所必需的研究挑战与方向。
原文摘要 · Abstract (English)
The operational capabilities and application domains of AI-enabled autonomous systems have expanded significantly in recent years due to advances in robotics and machine learning (ML). Demonstrating the safety of autonomous systems rigorously is critical for their responsible adoption but it is challenging as it requires robust methodologies that can handle novel and uncertain situations throughout the system lifecycle, including detecting out-of-distribution (OoD) data. Thus, OOD detection is receiving increased attention from the research, development and safety engineering communities. This comprehensive review analyses OOD detection techniques within the context of safety assurance for autonomous systems, in particular in safety-critical domains. We begin by defining the relevant concepts, investigating what causes OOD and exploring the factors which make the safety assurance of autonomous systems and OOD detection challenging. Our review identifies a range of techniques which can be used throughout the ML development lifecycle and we suggest areas within the lifecycle in which they may be used to support safety assurance arguments. We discuss a number of caveats that system and safety engineers must be aware of when integrating OOD detection into system lifecycles. We conclude by outlining the challenges and future work necessary for the safe development and operation of autonomous systems across a range of domains and applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。