用最优传输度量异常,让各类检测器在表格数据上更准更稳。
Calibrating Tabular Anomaly Detection via Optimal Transport
- 通过采样与聚类构建双分布,用最优传输距离衡量样本扰动程度。
- 在34个数据集上显著提升7种主流检测器性能,均值提升超15%。
- 无需调参,通用性强,适合实际部署的异常检测场景。
表格异常检测(TAD)因数据异质性而困难:特征无自然关联,分布与尺度差异大,类型多样。现有方法隐含假设异常模式,仅在特定数据集有效,缺乏普适性。本文提出CTAD(校准表格异常检测),一种模型无关的后处理框架,通过样本级校准增强任意已有TAD检测器。该方法利用随机采样生成的经验分布和K-means中心点构建的结构分布,以最优传输(OT)距离度量测试样本加入后的兼容性破坏程度。正常样本扰动小,异常样本扰动大,形成可放大的校准信号。理论证明OT距离下界与样本到中心点距离成正比,且异常样本期望校准分更高,解释其跨数据集泛化能力。在34个多样化表格数据集上,使用7种代表性的检测器(涵盖密度估计、分类、重构和孤立性方法)进行实验,结果表明CTAD在统计显著性上持续提升性能。尤为突出的是,即使对最先进的深度学习方法也有增益,且在不同超参数设置下表现稳健,无需额外调参即可部署。
原文摘要 · Abstract (English)
Tabular anomaly detection (TAD) remains challenging due to the heterogeneity of tabular data: features lack natural relationships, vary widely in distribution and scale, and exhibit diverse types. Consequently, each TAD method makes implicit assumptions about anomaly patterns that work well on some datasets but fail on others, and no method consistently outperforms across diverse scenarios. We present CTAD (Calibrating Tabular Anomaly Detection), a model-agnostic post-processing framework that enhances any existing TAD detector through sample-specific calibration. Our approach characterizes normal data via two complementary distributions, i.e., an empirical distribution from random sampling and a structural distribution from K-means centroids, and measures how adding a test sample disrupts their compatibility using Optimal Transport (OT) distance. Normal samples maintain low disruption while anomalies cause high disruption, providing a calibration signal to amplify detection. We prove that OT distance has a lower bound proportional to the test sample's distance from centroids, and establish that anomalies systematically receive higher calibration scores than normals in expectation, explaining why the method generalizes across datasets. Extensive experiments on 34 diverse tabular datasets with 7 representative detectors spanning all major TAD categories (density estimation, classification, reconstruction, and isolation-based methods) demonstrate that CTAD consistently improves performance with statistical significance. Remarkably, CTAD enhances even state-of-the-art deep learning methods and shows robust performance across diverse hyperparameter settings, requiring no additional tuning for practical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。