用不确定性评估公平性，选出最值得信赖的决策者
(Un)certainty of (Un)fairness: Preference-Based Selection of Certainly Fair Decision-Makers
- 引入贝叶斯方法量化决策差异的不确定性
- 相同偏差下，不确定性更低者被判定为更公平
- 适合需可解释公平性判断的系统设计与审核场景
公平性度量常用于评估机器学习模型及人类决策者在现实应用中的歧视与偏见，通常通过计算不同社会群体间概率结果的差异（如男女申请者的录取率差异）来实现。然而，传统公平性度量未考虑过程中的不确定性，且当两个决策者表现出相同差异时缺乏可比性。本文采用贝叶斯统计方法，量化差异的不确定性，将每个决策者（无论是模型还是人）表示为一个差异值及其对应的不确定性。通过定义决策者间的偏好关系，并使用暴力搜索方式依据效用函数选择最优决策者，最终得分最高的决策者可被解释为最确信其公平的决策者。
原文摘要 · Abstract (English)
Fairness metrics are used to assess discrimination and bias in decision-making processes across various domains, including machine learning models and human decision-makers in real-world applications. This involves calculating the disparities between probabilistic outcomes among social groups, such as acceptance rates between male and female applicants. However, traditional fairness metrics do not account for the uncertainty in these processes and lack of comparability when two decision-makers exhibit the same disparity. Using Bayesian statistics, we quantify the uncertainty of the disparity to enhance discrimination assessments. We represent each decision-maker, whether a machine learning model or a human, by its disparity and the corresponding uncertainty in that disparity. We define preferences over decision-makers and utilize brute-force to choose the optimal decision-maker according to a utility function that ranks decision-makers based on these preferences. The decision-maker with the highest utility score can be interpreted as the one for whom we are most certain that it is fair.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。