考虑因果图不确定性，更可靠地评估模型偏见。
Counterfactual Fairness with Graph Uncertainty
- 用领域知识约束生成多个可能的因果图
- 通过熵值量化因果图不确定性，给出偏见指标置信区间
- 在真实数据上验证了常见偏见，即使知识有限也有效
评估机器学习模型的偏见是构建可信、稳健系统的关键。反事实公平性（CF）审计利用因果框架衡量模型偏见，但其结论依赖于单一且通常不确定的因果图。本文提出反事实公平性与图不确定性（CF-GU），将因果图的不确定性纳入评估过程：(i) 在领域知识约束下通过自助法生成一组合理的有向无环图（DAG）；(ii) 用归一化香农熵量化图的不确定性；(iii) 提供反事实公平性指标的置信边界。合成数据实验表明，不同领域假设可支持或反驳公平性审计；真实数据实验（COMPAS和Adult数据集）在仅提供少量领域知识时，仍以高置信度识别出已知偏见。
原文摘要 · Abstract (English)
Evaluating machine learning (ML) model bias is key to building trustworthy and robust ML systems. Counterfactual Fairness (CF) audits allow the measurement of bias of ML models with a causal framework, yet their conclusions rely on a single causal graph that is rarely known with certainty in real-world scenarios. We propose CF with Graph Uncertainty (CF-GU), a bias evaluation procedure that incorporates the uncertainty of specifying a causal graph into CF. CF-GU (i) bootstraps a Causal Discovery algorithm under domain knowledge constraints to produce a bag of plausible Directed Acyclic Graphs (DAGs), (ii) quantifies graph uncertainty with the normalized Shannon entropy, and (iii) provides confidence bounds on CF metrics. Experiments on synthetic data show how contrasting domain knowledge assumptions support or refute audits of CF, while experiments on real-world data (COMPAS and Adult datasets) pinpoint well-known biases with high confidence, even when supplied with minimal domain knowledge constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。