arXiv:2409.10951cs.LG2024-09被引 13

解决异常检测中少数群体被误判的问题,提升公平性。

Fair Anomaly Detection For Imbalanced Groups

  • 用对比学习和重加权自编码器平衡数据分布,兼顾少数群体。
  • 在真实数据集上显著降低少数群体的误报率,提升公平性指标。
  • 适合金融反欺诈、网络安全等存在群体不均衡的场景。

异常检测在金融反欺诈、网络安全等领域广泛应用。由于保护群体与非保护群体之间数据极度不平衡,且正常样本与异常样本分布不均,现有方法往往只关注占主导地位的非保护群体,导致对保护群体的异常检测不公平。已有公平性方法在非保护群体占多数时,会错误地将大量保护群体的正常样本判定为异常。这是由于学习目标过于聚焦常见模式而忽视了少数群体。为此,我们提出 FairAD,一种面向不平衡场景的公平异常检测方法,包含公平感知对比学习模块和重加权自编码器模块,分别保证公平性和处理数据不平衡问题。理论分析表明,所提出的对比学习正则化可保障群体公平性。实证研究显示,FairAD 在多个真实数据集上均表现出高效且有效的性能。

原文摘要 · Abstract (English)

Anomaly detection (AD) has been widely studied for decades in many real-world applications, including fraud detection in finance, and intrusion detection for cybersecurity, etc. Due to the imbalanced nature between protected and unprotected groups and the imbalanced distributions of normal examples and anomalies, the learning objectives of most existing anomaly detection methods tend to solely concentrate on the dominating unprotected group. Thus, it has been recognized by many researchers about the significance of ensuring model fairness in anomaly detection. However, the existing fair anomaly detection methods tend to erroneously label most normal examples from the protected group as anomalies in the imbalanced scenario where the unprotected group is more abundant than the protected group. This phenomenon is caused by the improper design of learning objectives, which statistically focus on learning the frequent patterns (i.e., the unprotected group) while overlooking the under-represented patterns (i.e., the protected group). To address these issues, we propose FairAD, a fairness-aware anomaly detection method targeting the imbalanced scenario. It consists of a fairness-aware contrastive learning module and a rebalancing autoencoder module to ensure fairness and handle the imbalanced data issue, respectively. Moreover, we provide the theoretical analysis that shows our proposed contrastive learning regularization guarantees group fairness. Empirical studies demonstrate the effectiveness and efficiency of FairAD across multiple real-world datasets.

异常检测公平性不平衡数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。