让自动驾驶感知系统像人一样理解场景关系，提升安全可靠性。
Assured Autonomy with Neuro-Symbolic Perception
- 用神经符号方法融合感知与推理，构建带语义关系的场景图。
- 在仿真与真实数据上验证，显著提升对复杂场景的理解能力。
- 适合关注自动驾驶、机器人安全性的研究者与工程师。
许多现役的先进人工智能模型在信息物理系统（CPS）中虽具备高精度，但本质上只是模式匹配器，缺乏安全保证，难以应用于高风险或对抗性环境。为推动可信人工智能发展，我们倡导一种范式转变：将符号结构注入数据驱动的感知模型，借鉴人类从低层特征与高层上下文进行推理的能力。为此，提出神经符号感知框架（NeuSPaPer），通过联合物体检测与场景图生成（SGG），实现深层场景理解。该框架利用基础模型离线提取知识，结合专用的SGG算法实现实时部署，构建结构化关系图，确保自主系统情境感知的完整性。在基于物理的模拟器和真实世界数据集上，验证了SGG如何弥合底层传感感知与高层推理之间的鸿沟，为构建韧性、上下文感知的人工智能奠定基础，推动信息物理系统中的可信自主。
原文摘要 · Abstract (English)
Many state-of-the-art AI models deployed in cyber-physical systems (CPS), while highly accurate, are simply pattern-matchers.~With limited security guarantees, there are concerns for their reliability in safety-critical and contested domains. To advance assured AI, we advocate for a paradigm shift that imbues data-driven perception models with symbolic structure, inspired by a human's ability to reason over low-level features and high-level context. We propose a neuro-symbolic paradigm for perception (NeuSPaPer) and illustrate how joint object detection and scene graph generation (SGG) yields deep scene understanding.~Powered by foundation models for offline knowledge extraction and specialized SGG algorithms for real-time deployment, we design a framework leveraging structured relational graphs that ensures the integrity of situational awareness in autonomy. Using physics-based simulators and real-world datasets, we demonstrate how SGG bridges the gap between low-level sensor perception and high-level reasoning, establishing a foundation for resilient, context-aware AI and advancing trusted autonomy in CPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。