提出公平投影方法,让异常检测对不同群体更公正。
Fairness-aware Anomaly Detection via Fair Projection
- 从正常数据学习统一投影,使各群体数据分布一致
- 在真实数据集上实现准确率与公平性的更好平衡
- 无需手动设阈值,全局评估模型公平性
无监督异常检测在金融、医疗、社交媒体和网络安全等高社会影响领域至关重要,常涉及年龄、性别、种族、疾病等人口统计信息。此类场景中,异常检测系统可能引入偏见,导致不同群体受到不公平对待,甚至加剧社会偏见。本文首先系统分析了在无监督异常检测中实现群体公平性的可行性及必要假设。其次提出一种新的公平感知异常检测方法 FairAD。该方法从正常训练数据中学习一个投影,将不同人口统计群体的数据映射到一个简单且紧凑的共同目标分布,从而为数据密度估计提供可靠基础。异常可通过密度直接识别,而共同目标分布确保了群体间公平性。此外,提出一种无需阈值的全局公平性度量,避免对人工阈值的依赖。在多个真实世界基准测试上,该方法在群体数据平衡与不平衡情况下均实现了检测准确率与公平性的更优权衡。
原文摘要 · Abstract (English)
Unsupervised anomaly detection is a critical task in many high-social-impact applications such as finance, healthcare, social media, and cybersecurity, where demographics involving age, gender, race, disease, etc, are used frequently. In these scenarios, possible bias from anomaly detection systems can lead to unfair treatment for different groups and even exacerbate social bias. In this work, first, we thoroughly analyze the feasibility and necessary assumptions for ensuring group fairness in unsupervised anomaly detection. Second, we propose a novel fairness-aware anomaly detection method FairAD. From the normal training data, FairAD learns a projection to map data of different demographic groups to a common target distribution that is simple and compact, and hence provides a reliable base to estimate the density of the data. The density can be directly used to identify anomalies while the common target distribution ensures fairness between different groups. Furthermore, we propose a threshold-free fairness metric that provides a global view for model's fairness, eliminating dependence on manual threshold selection. Experiments on real-world benchmarks demonstrate that our method achieves an improved trade-off between detection accuracy and fairness under both balanced and skewed data across different groups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。