自动挑选最适合的异常检测模型,无需标注数据
MetaOOD: Automatic Selection of OOD Detection Models
- 用元学习分析历史表现,自动匹配最优检测模型
- 24组测试中比11种基线方法表现更好,显著提升准确率
- 适合无标签场景,如自动驾驶和医疗诊断等关键领域
如何为不同任务自动选择合适的分布外(OOD)检测模型?这对保障开放世界应用的可靠性至关重要,尤其在在线交易、自动驾驶和实时患者诊断等关键领域。尽管已有多种OOD检测方法,但针对多样化任务选择最优模型的问题仍缺乏研究,尤其在无真实标签的场景下。本文提出MetaOOD,首个零样本、无监督的元学习框架,利用历史性能数据自动选择适合新数据集的检测模型,无需测试时提供标签。为更精准量化任务相似性,引入基于语言模型的嵌入,捕捉数据集与模型的特有OOD特征。通过24个测试数据集对和11种检测模型的广泛实验,验证了MetaOOD显著优于现有方法,仅带来微小时间开销。经威科克森统计检验,其性能超越包括主流检测器和先进无监督选择方法在内的11个基线。
原文摘要 · Abstract (English)
How can we automatically select an out-of-distribution (OOD) detection model for various underlying tasks? This is crucial for maintaining the reliability of open-world applications by identifying data distribution shifts, particularly in critical domains such as online transactions, autonomous driving, and real-time patient diagnosis. Despite the availability of numerous OOD detection methods, the challenge of selecting an optimal model for diverse tasks remains largely underexplored, especially in scenarios lacking ground truth labels. In this work, we introduce MetaOOD, the first zero-shot, unsupervised framework that utilizes meta-learning to select an OOD detection model automatically. As a meta-learning approach, MetaOOD leverages historical performance data of existing methods across various benchmark OOD detection datasets, enabling the effective selection of a suitable model for new datasets without the need for labeled data at the test time. To quantify task similarities more accurately, we introduce language model-based embeddings that capture the distinctive OOD characteristics of both datasets and detection models. Through extensive experimentation with 24 unique test dataset pairs to choose from among 11 OOD detection models, we demonstrate that MetaOOD significantly outperforms existing methods and only brings marginal time overhead. Our results, validated by Wilcoxon statistical tests, show that MetaOOD surpasses a diverse group of 11 baselines, including established OOD detectors and advanced unsupervised selection methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。