arXiv:2607.16811cs.LG2026-07

提出可解释的流数据污染检测方法,自动识别异常模式并说明原因。

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

  • 用高斯混合模型定义数据模式,未匹配模式的比例作为漂移信号。
  • 修复高维下检测失效问题,64维时准确率从0.5升至0.89。
  • 既能检测漂移又可指出数据离开哪个模式及偏离程度,适合监控场景。

现有漂移检测器要么无效,要么无法解释;而可解释的检测器在高维下表现差。本文针对高斯混合模型(GMM)提出一种新方法:每个拟合分量对应一个“状态”,窗口中未被任何状态覆盖的比例即为未解释质量,既是漂移信号也是其解释。我们揭示该统计量在高维下失效的原因——当维度d>9时,几乎所有点都超过3倍标准差距离,导致卫星数据集(d=36)和Optdigits(d=64)上的窗口级ROC-AUC降至0.5。通过将半径校准为sqrt(chi-squared_d(0.99)),性能恢复至AUC 1.00和0.89,且低维不变。在七个公开基准、五次随机种子、八类无模型检测器中,该方法在10%污染率下于五个数据集最优或并列最优(仅在Pendigits和Optdigits略逊),稀疏污染时表现更稳定:2%污染下平均AUC达0.86,优于任一其他无模型检测器(最高0.73)。唯一能报告数据离开的具体状态及其偏离程度。明确其局限性:仅对新状态漂移敏感,对已知状态重加权不敏感;密度模型的极值理论校准误报率在d>36后下降。全部代码与实验均已开源。

原文摘要 · Abstract (English)

Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension. We close that gap for Gaussian mixture models (GMMs): each fitted component is a named "regime," and the fraction of a stream window matching no regime -- its unexplained mass -- is a drift signal that is simultaneously its own explanation. We identify why this statistic collapses in high dimension and repair it. Under a correct component a normal point in d dimensions lies about sqrt(d) sigma from the mean, so once d exceeds 9 essentially every point exceeds a fixed 3-sigma radius: window-level ROC-AUC is exactly 0.50 on Satellite (d=36) and Optdigits (d=64). Calibrating the radius to sqrt(chi-squared_d(0.99)) removes the collapse -- AUC 1.00 and 0.89 -- while leaving low dimensions unchanged. Across seven public benchmarks, five seeds, and eight model-free detectors spanning the kernel, classifier, projection, density-difference, transport, likelihood and partition families, the repaired statistic is best or tied-best on five of seven datasets at 10% window contamination (its two losses are Pendigits, where the whole field beats it, and Optdigits), and as contamination becomes sparse the sample-level detectors fade toward chance while it degrades most gracefully: at 2% its mean AUC across the benchmarks is 0.86 against at most 0.73 for any model-free detector (1.00 vs. MMD's 0.72 on KDD-http) -- while alone among them reporting which regime the data left and how far outside it the window lies. We delimit its scope honestly: unexplained mass detects and explains novel-regime drift but is blind by construction to in-support re-weighting of known regimes, where distribution-level tests are required and explain nothing; and the underlying density model's EVT-calibrated false-alarm rates degrade above d of about 36. All code and experiments are released.

数据漂移可解释性高维检测流数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。