用因果关系引导测试样本生成,显著提升公平性漏洞检测能力
Causally Perturbed Fairness Testing
- 基于因果推断找出与敏感特征直接相关的非敏感特征
- 在1296个案例中使93%的基线生成器更有效发现公平性缺陷
- 适合关注AI模型公平性测试的研究者和工程师
为缓解敏感特征(如性别、年龄或种族)带来的不公平与不道德歧视,公平性测试在处理表格数据的AI系统中至关重要。现有方法多聚焦于测试样本生成器的设计,忽视了数据特征中可指导扰动的关键信息,限制了其潜力。本文提出通用框架CausalFT,通过因果推断提取与敏感特征具有直接因果关联的非敏感特征,该特征能共同影响标签预测。此因果关系被无缝注入扰动过程以指导测试样本生成器。不同于仅在生成器层面改进的方法,CausalFT作为高层框架可适配多种基线生成器。在1296个案例上的实验表明,CausalFT可显著提升任意基线生成器在93%案例中发现公平性缺陷的能力,且额外运行开销可控。相较于仅依赖相关性排序非敏感特征的先进方法,CausalFT在64%案例中表现更优且效率更高。此外,它在几乎所有情况下均增强了模型对偏见的鲁棒性。
原文摘要 · Abstract (English)
To mitigate unfair and unethical discrimination over sensitive features (e.g., gender, age, or race), fairness testing plays an integral role in engineering systems that leverage AI models to handle tabular data. A key challenge therein is how to effectively reveal fairness bugs under an intractable sample size using perturbation. Much current work has been focusing on designing the test sample generators, ignoring the valuable knowledge about data characteristics that can help guide the perturbation and hence limiting their full potential. In this paper, we seek to bridge such a gap by proposing a generic framework of causally perturbed fairness testing, dubbed CausalFT. Through causal inference, the key idea of CausalFT is to extract the most directly and causally relevant non-sensitive feature to its sensitive counterpart, which can jointly influence the prediction of the label. Such a causal relationship is then seamlessly injected into the perturbation to guide a test sample generator. Unlike existing generator-level work, CausalFT serves as a higher-level framework that can be paired with diverse base generators. Extensive experiments on 1296 cases confirm that CausalFT can considerably improve arbitrary base generators in revealing fairness bugs over 93% of the cases with acceptable extra runtime overhead. Compared with a state-of-the-art approach that ranks the non-sensitive features solely based on correlation, CausalFT performs significantly better on 64% cases while being much more efficient. Further, CausalFT can better improve bias resilience in nearly all cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。