arXiv:2411.08227cs.CVcs.AI2024-11被引 34

动态更新类别中心,提升多模态异常检测精度

DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection

  • 根据批次内样本相似性动态调整类别中心
  • 在远端异常检测中性能提升最高达80%
  • 适用于多模态模型的通用增强框架

分布外(OOD)检测对于保障机器学习模型鲁棒性至关重要,能识别偏离训练数据分布的样本。传统方法主要针对单模态输入,而近年多模态模型展现出利用视频、光流、音频等多源信息提升检测性能的潜力。然而,现有方法常忽视类内数据的内部变异性,假设同类别样本完全一致,导致预测差异被均匀放大,影响性能。为此,本文提出动态原型更新(DPU)框架,通过测量每批内相似样本的方差,动态更新每个类别的中心表示,实现自适应调整。该方法基于更新后的类别中心放大预测差异,显著提升模型在不同模态下的鲁棒性与泛化能力。在两个任务、五个数据集和九种基础OOD算法上的大量实验表明,DPU显著提升检测性能,远端异常检测最高提升80%,刷新多模态OOD检测新纪录。代码已公开于GitHub,便于复现与使用。

原文摘要 · Abstract (English)

Out-of-distribution (OOD) detection is essential for ensuring the robustness of machine learning models by identifying samples that deviate from the training distribution. While traditional OOD detection has primarily focused on single-modality inputs, such as images, recent advances in multimodal models have demonstrated the potential of leveraging multiple modalities (e.g., video, optical flow, audio) to enhance detection performance. However, existing methods often overlook intra-class variability within in-distribution (ID) data, assuming that samples of the same class are perfectly cohesive and consistent. This assumption can lead to performance degradation, especially when prediction discrepancies are uniformly amplified across all samples. To address this issue, we propose Dynamic Prototype Updating (DPU), a novel plug-and-play framework for multimodal OOD detection that accounts for intra-class variations. Our method dynamically updates class center representations for each class by measuring the variance of similar samples within each batch, enabling adaptive adjustments. This approach allows us to amplify prediction discrepancies based on the updated class centers, thereby improving the model's robustness and generalization across different modalities. Extensive experiments on two tasks, five datasets, and nine base OOD algorithms demonstrate that DPU significantly improves OOD detection performance, setting a new state-of-the-art in multimodal OOD detection, with improvements of up to 80 percent in Far-OOD detection. To facilitate accessibility and reproducibility, our code is publicly available on GitHub.

多模态异常检测动态更新鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。