提出可视化基准测试与两种改进方法，提升模型对分布外数据的检测精度。
phepy: Visual benchmarks and improvements for out-of-distribution detectors
- 设计三个直观的玩具案例，检验检测器识别线性、非线性及高维稀疏子空间的能力。
- 引入t- poking和OOD样本加权，显著改善监督式检测器在边界处的准确性。
- 适合关注模型鲁棒性与安全性的研究人员参考使用。
将机器学习应用于高维、稀疏或有偏的训练数据时，模型在分布外（OOD）输入上可能产生无效预测且误差无界。由于真实数据集难以用于测试OOD检测方法，本文设计了一个包含三个新颖且易于可视化的玩具案例的基准测试。这些案例可直接评估检测器是否能识别（1）线性概念、（2）非线性概念、以及（3）高维空间中的细小分布内子空间（如“针在干草堆中”）。我们利用该基准评估了多种文献方法的性能。鉴于真实感分布外样本可能有助于提升检测效果，我们还回顾了几种简单的合成方法以生成监督训练所需的分布外样本。在此基础上，提出两种改进：t-poking与分布外样本加权，使监督式检测器在分布内与分布外边界更加精确，尤其适用于真实分布内与合成分布外样本导致决策边界模糊的情况。最后，给出构建与应用分布外检测器的实用建议。
原文摘要 · Abstract (English)
Applying machine learning to increasingly high-dimensional problems with sparse or biased training data increases the risk that a model is used on inputs outside its training domain. For such out-of-distribution (OOD) inputs, the model can no longer make valid predictions, and its error is potentially unbounded. Since testing OOD detection methods on real-world datasets is complicated, we design a benchmark for OOD detection, which includes three novel and easily-visualisable toy examples. These simple examples provide direct and intuitive insight into whether the detector is able to detect (1) linear and (2) non-linear concepts and (3) identify thin in-distribution (ID) subspaces (needles) within high-dimensional spaces (haystacks). We use our benchmark to evaluate the performance of various methods from the literature. Since tactile examples of OOD inputs may benefit OOD detection, we also review several simple methods to synthesise OOD inputs for supervised training. We introduce two improvements, $t$-poking and OOD sample weighting, to make supervised detectors more precise at the ID-OOD boundary. This is especially important when conflicts between real ID and synthetic OOD sample blur the decision boundary. Finally, we provide recommendations for constructing and applying OOD detectors in machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。