让分类器在所有阈值下对不同群体都公平，解决算法偏见问题。
FROC: Building Fair ROC from a Trained Classifier
- 通过后处理将原始分类器输出转为随机化公平分类器。
- 理论证明最小化AUC损失可实现ε₁-等公平ROC，且可达到最优。
- 提出FROC算法，线性时间完成，适用于真实数据集中的多种模型。
本文研究具有二元保护群体的公平概率二分类问题。分类器输出得分，从业者根据期望的假阳性与假阴性权衡选择阈值，基于分类器的ROC曲线确定。然而，所得分类器可能对某一保护群体不公平。理想情况是：无论采用何种阈值，分类器对两个保护群体均应公平，即两组的假正率（FPR）与真正率（TPR）之间的L_p范数不超过ε。我们称这种在两类保护属性上满足的公平性为ε_p-等公平ROC。给定一个不满足ε₁-等公平ROC的分类器，本文设计一种后处理方法，将其输出（得分）转换为一种随机化但公平的分类器，使其满足ε₁-等公平ROC。首先引入每个保护群体在ROC曲线上进行阈值查询的模型。结果分类器必然面临AUC下降。在此模型下，本文提供严格的理论分析，推导出实现ε₁-等公平ROC所需的最小AUC损失。为此，设计了线性时间算法FROC，将原分类器输出转化为满足ε₁-等公平ROC的随机分类器。在特定理论条件下，证明了FROC能达到理论最优保证。同时，在多个真实世界数据集和多种训练分类器上评估了FROC的性能。
原文摘要 · Abstract (English)
This paper considers the problem of fair probabilistic binary classification with binary protected groups. The classifier assigns scores, and a practitioner predicts labels using a certain cut-off threshold based on the desired trade-off between false positives vs. false negatives. It derives these thresholds from the ROC of the classifier. The resultant classifier may be unfair to one of the two protected groups in the dataset. It is desirable that no matter what threshold the practitioner uses, the classifier should be fair to both the protected groups; that is, the $\mathcal{L}_p$ norm between FPRs and TPRs of both the protected groups should be at most $\varepsilon$. We call such fairness on ROCs of both the protected attributes $\varepsilon_p$-Equalized ROC. Given a classifier not satisfying $\varepsilon_1$-Equalized ROC, we aim to design a post-processing method to transform the given (potentially unfair) classifier's output (score) to a suitable randomized yet fair classifier. That is, the resultant classifier must satisfy $\varepsilon_1$-Equalized ROC. First, we introduce a threshold query model on the ROC curves for each protected group. The resulting classifier is bound to face a reduction in AUC. With the proposed query model, we provide a rigorous theoretical analysis of the minimal AUC loss to achieve $\varepsilon_1$-Equalized ROC. To achieve this, we design a linear time algorithm, namely \texttt{FROC}, to transform a given classifier's output to a probabilistic classifier that satisfies $\varepsilon_1$-Equalized ROC. We prove that under certain theoretical conditions, \texttt{FROC}\ achieves the theoretical optimal guarantees. We also study the performance of our \texttt{FROC}\ on multiple real-world datasets with many trained classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。