提出可适配真实场景的公平性评估框架，解决方法间不可比问题。
ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methods
- 构建可灵活适配不同公平性设定的评测框架
- 在多类数据集上验证，避免公平性与准确率权衡
- 适合希望公正比较各类公平性方法的研究者
众多方法致力于通过缓解机器学习中的偏见来实现对敏感特征的公平性。然而，各方法所面对的问题设定差异显著，包括干预阶段、敏感特征构成、公平性定义及输出分布等。即使在二分类任务中，这些细微差别也使得公平性方法的基准测试极为复杂，其性能高度依赖于偏见缓解问题的具体设定。为此，我们提出 ABCFair，一种可适应真实世界问题设定的基准评估方法，使各类方法在任意应用场景下均可进行合理比较。我们在大规模传统数据集及双标签（有偏与无偏）数据集上，对预处理、内处理和后处理方法进行了应用，有效规避了公平性与准确率之间的权衡。
原文摘要 · Abstract (English)
Numerous methods have been implemented that pursue fairness with respect to sensitive features by mitigating biases in machine learning. Yet, the problem settings that each method tackles vary significantly, including the stage of intervention, the composition of sensitive features, the fairness notion, and the distribution of the output. Even in binary classification, these subtle differences make it highly complicated to benchmark fairness methods, as their performance can strongly depend on exactly how the bias mitigation problem was originally framed. Hence, we introduce ABCFair, a benchmark approach which allows adapting to the desiderata of the real-world problem setting, enabling proper comparability between methods for any use case. We apply ABCFair to a range of pre-, in-, and postprocessing methods on both large-scale, traditional datasets and on a dual label (biased and unbiased) dataset to sidestep the fairness-accuracy trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。