反其道而行之:用可控过拟合提升异常检测精度
Friend or Foe? Harnessing Controllable Overfitting for Anomaly Detection
- 设计可调控过拟合机制,通过新指标识别最优过拟合区间
- 在多类和单类异常检测任务中均达到最新最优性能
- 适合关注模型敏感性与泛化权衡的研究者和工程实践者
过拟合传统上被视为异常检测中的负面因素,过度泛化会削弱模型对细微异常的敏感性。本文提出可控过拟合异常检测框架(COAD),挑战这一观点,通过战略性利用过拟合增强异常判别能力。引入异常保留率(ARQ)量化过拟合程度,识别出使模型对异常最敏感但不牺牲泛化的‘黄金过拟合区间’。进一步提出相对异常分布指数(RADI),优于传统AUROC,显式建模正常与异常得分分布的分离度。理论分析表明,RADI结合ARQ可系统追踪过拟合动态对检测性能的影响。同时严格验证高斯噪声作为伪异常生成器的统计有效性,强化方法普适性。实证结果表明,该方法在单类与多类异常检测任务中均达到当前最优水平,重新定义过拟合为强大策略而非缺陷。
原文摘要 · Abstract (English)
Overfitting has traditionally been viewed as detrimental to anomaly detection, where excessive generalization often limits models' sensitivity to subtle anomalies. Our work challenges this conventional view by introducing Controllable Overfitting-based Anomaly Detection (COAD), a novel framework that strategically leverages overfitting to enhance anomaly discrimination capabilities. We propose the Aberrance Retention Quotient (ARQ), a novel metric that systematically quantifies the extent of overfitting, enabling the identification of an optimal golden overfitting interval wherein model sensitivity to anomalies is maximized without sacrificing generalization. To comprehensively capture how overfitting affects detection performance, we further propose the Relative Anomaly Distribution Index (RADI), a metric superior to traditional AUROC by explicitly modeling the separation between normal and anomalous score distributions. Theoretically, RADI leverages ARQ to track and evaluate how overfitting impacts anomaly detection, offering an integrated approach to understanding the relationship between overfitting dynamics and model efficacy. We also rigorously validate the statistical efficacy of Gaussian noise as pseudo-anomaly generators, reinforcing the method's broad applicability. Empirical evaluations demonstrate that our controllable overfitting method achieves State-Of-The-Art(SOTA) performance in both one-class and multi-class anomaly detection tasks, thus redefining overfitting as a powerful strategy rather than a limitation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。