用事件相机与摄像头融合，实现自动驾驶毫秒级异常检测。
When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Network
- 异步图网络处理事件流，CNN提取图像空间特征，双模态协同
- 在基准数据集上响应时间达毫秒级，准确率优于现有方法
- 适合对实时性要求高的自动驾驶安全系统
异常检测对自动驾驶系统的安全与可靠性至关重要。现有方法多关注检测准确率，忽视了在时间敏感的驾驶场景中响应时间的重要性。本文提出一种面向自动驾驶的实时异常检测方法，同时兼顾最低响应时间和高准确率。我们设计了一种新颖的多模态异步混合网络，融合事件相机产生的事件流与RGB相机的图像数据。网络通过异步图神经网络利用事件相机的高时间分辨率，同时结合卷积神经网络（CNN）从RGB图像中提取的空间特征。该方法有效捕捉了驾驶环境的时间动态与空间细节，实现快速精准的异常检测。大量实验表明，该方法在基准数据集上同时优于现有技术的准确率和响应时间，达到毫秒级实时性能。
原文摘要 · Abstract (English)
Anomaly detection is essential for the safety and reliability of autonomous driving systems. Current methods often focus on detection accuracy but neglect response time, which is critical in time-sensitive driving scenarios. In this paper, we introduce real-time anomaly detection for autonomous driving, prioritizing both minimal response time and high accuracy. We propose a novel multimodal asynchronous hybrid network that combines event streams from event cameras with image data from RGB cameras. Our network utilizes the high temporal resolution of event cameras through an asynchronous Graph Neural Network and integrates it with spatial features extracted by a CNN from RGB images. This combination effectively captures both the temporal dynamics and spatial details of the driving environment, enabling swift and precise anomaly detection. Extensive experiments on benchmark datasets show that our approach outperforms existing methods in both accuracy and response time, achieving millisecond-level real-time performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。