arXiv:2506.05171eess.SYcs.AI2025-06被引 2

提出可证明的概率安全框架,让智能系统在复杂环境中更安全地规模化应用。

Towards provable probabilistic safety for scalable embodied AI systems

  • 用概率边界替代全场景确定性验证,兼顾理论保障与实际可行性
  • 通过统计方法构建可渐进逼近的安全边界,支持大规模部署
  • 适合自动驾驶、医疗机器人等高风险场景的开发者参考

具身智能系统(由AI模型与物理装置组成)在多个领域日益普及。由于系统故障罕见且场景复杂,确保其在复杂环境中的安全性仍是重大挑战,严重制约了其在自动驾驶、医疗设备和机器人等安全关键领域的规模化部署。虽然理论上实现对所有可能场景的确定性安全验证是理想目标,但极端案例的稀有性和复杂性使其对可扩展的具身AI系统不切实际。因此,当前多采用经验性安全评估,但缺乏可证明的保证,存在显著局限。本文主张转向可证明的概率安全范式,将可证明的保证与向整体性能的概率安全边界渐进逼近相结合。该新范式更好利用统计方法提升可行性与可扩展性,明确的概率安全边界使具身AI系统得以规模化部署。本文提出可证明概率安全的路线图,并讨论相应挑战与潜在解决方案,为安全关键应用中具身AI系统的安全规模化提供可行路径。

原文摘要 · Abstract (English)

Embodied AI systems, comprising AI models and physical plants, are increasingly prevalent across various applications. Due to the rarity of system failures, ensuring their safety in complex operating environments remains a major challenge, which severely hinders their large-scale deployment in safety-critical domains, such as autonomous vehicles, medical devices, and robotics. While achieving provable deterministic safety-verifying system safety across all possible scenarios-remains theoretically ideal, the rarity and complexity of corner cases make this approach impractical for scalable embodied AI systems. Instead, empirical safety evaluation is employed as an alternative, but the absence of provable guarantees imposes significant limitations. To address these issues, we argue for a paradigm shift to provable probabilistic safety that integrates provable guarantees with progressive achievement toward a probabilistic safety boundary on overall system performance. The new paradigm better leverages statistical methods to enhance feasibility and scalability, and a well-defined probabilistic safety boundary enables embodied AI systems to be deployed at scale. In this Perspective, we outline a roadmap for provable probabilistic safety, along with corresponding challenges and potential solutions. By bridging the gap between theoretical safety assurance and practical deployment, this Perspective offers a pathway toward safer, large-scale adoption of embodied AI systems in safety-critical applications.

具身智能概率安全系统可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。