用简单方法合成多模态异常数据,提升模型对未知情况的识别能力。
Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
- 通过特征混合生成多模态异常样本,无需复杂训练。
- 在多个数据集上实现顶尖性能,速度提升10到370倍。
- 适合自动驾驶等安全关键场景的异常检测需求。
分布外(OOD)检测与分割对于自动驾驶、机器人辅助手术等安全关键应用至关重要。现有研究多聚焦于单模态图像数据,而真实场景是多模态的,需融合多种模态以提升检测效果。主要挑战在于未知数据缺乏监督信号,导致模型对OOD样本过度自信。为此,我们提出特征混合(Feature Mixing)方法,一种极简且快速的多模态异常合成技术,具有理论支持,可进一步优化以更好区分分布内(ID)与分布外(OOD)数据。该方法具备模态无关性,适用于多种模态组合。此外,我们构建了全新多模态数据集CARLA-OOD,包含多样场景与天气下的合成异常物体。在SemanticKITTI、nuScenes、CARLA-OOD及MultiOOD基准上的大量实验表明,特征混合达到当前最优性能,速度提升达10至370倍。源代码与数据集将开源于https://github.com/mona4399/FeatureMixing。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused on unimodal image data, real-world applications are inherently multimodal, requiring the integration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions on OOD samples. To address this challenge, we propose Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis with theoretical support, which can be further optimized to help the model better distinguish between in-distribution (ID) and OOD data. Feature Mixing is modality-agnostic and applicable to various modality combinations. Additionally, we introduce CARLA-OOD, a novel multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Extensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-art performance with a $10 \times$ to $370 \times$ speedup. Our source code and dataset will be available at https://github.com/mona4399/FeatureMixing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。