用可解释的统计方法替代人工设阈值,让异常检测结果更可信。
Conformal Anomaly Detection in Python: Moving Beyond Heuristic Thresholds with nonconform
- 将异常得分转为有统计意义的p值,实现严格校准
- 支持多种校准策略,包括高效且抗数据偏移的扩展方法
- 兼容scikit-learn等主流工具,适合科研与工程落地
大多数异常检测系统输出的是得分而非可校准的决策,导致从业者需凭经验设定阈值,缺乏明确的统计解释。共形异常检测通过将异常得分转化为在数据可交换性假设下有效的校准p值,解决了这一问题,已有研究不断拓展其适用场景。本文提出nonconform——一个用于在现有机器学习流程中应用共形异常检测的Python包,并以此为基础提供实践导向的领域入门介绍。该包与scikit-learn、PyOD及自定义检测器集成,提供统一接口以实现校准、p值生成与假阳性率控制。支持从简单分割共形化到更高效、具备偏移感知能力的扩展策略。通过从基础概念到高级共形化策略的递进式讲解,辅以代码示例,本文将共形异常检测背后的统计思想与其实际应用紧密结合。实证结果表明,所实现方法能实现统计上严谨的异常检测。整体而言,该包与阐述旨在使核心共形异常检测工作流在实验与生产环境中更易访问和复现。
原文摘要 · Abstract (English)
Most anomaly detection systems output scores rather than calibrated decisions, leaving practitioners to choose thresholds heuristically and without clear statistical interpretation. Conformal anomaly detection addresses this limitation by converting anomaly scores into calibrated p-values that are valid under the statistical assumption of data exchangeability, with a growing literature extending this idea beyond that setting. We present nonconform, a Python package for applying conformal anomaly detection within existing machine-learning workflows, and use it as the basis for an implementation-grounded introduction to the field. The package integrates with scikit-learn, PyOD, and custom anomaly detectors, and provides a unified interface for calibration, p-value generation, and false discovery rate control. It supports several conformalization strategies, ranging from simple split-conformal calibration to more data-efficient and shift-aware extensions. Through a progression from foundational concepts to advanced conformalization strategies, complemented by code examples, the paper connects the statistical ideas behind conformal anomaly detection to their practical use in nonconform. Empirical results demonstrate that the implemented methods enable statistically principled anomaly detection. Together, the package and exposition aim to make core conformal anomaly detection workflows more accessible and reproducible in experimental and production-oriented settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。