arXiv:2510.03734cs.LGcs.AI2025-10中稿 · NeurIPS

在只知通过者真实结果时,高效审计分类器公平性。

Cost Efficient Fairness Audit Under Partial Feedback

  • 基于截断样本设计新成本模型,更贴近真实审计开销。
  • 黑箱与混合分布两种设定下,算法成本比基线低50%以上。
  • 适用于多种公平性指标,适合金融、招聘等高成本场景。

我们研究在部分反馈下的分类器公平性审计问题,即仅能获取被正向分类个体的真实标签(如仅批准申请人有还款记录)。为此,我们提出一种新型成本模型,更准确反映信用评估、贷款处理及潜在违约等实际开销。目标是设计比随机探索和自然基线更节省成本的最优审计算法。本文考虑两种场景:无数据分布假设的黑箱模型,以及特征与真实标签服从指数族混合分布的混合模型。在黑箱设定下,我们提出一个近似最优的审计算法,并证明自然基线可能严格劣于最优解;在混合模型中,我们设计的新算法显著降低审计成本。方法结合了截断样本学习与最大后验查询机制,将球形高斯混合结果推广至指数族混合,或具独立价值。算法适用于主流公平性度量如群体平等、机会平等和均等几率。实验表明,在Adult Income和Law School等真实数据集上,算法性能持续优于基线约50%的审计成本。

原文摘要 · Abstract (English)

We study the problem of auditing the fairness of a given classifier under partial feedback, where true labels are available only for positively classified individuals, (e.g., loan repayment outcomes are observed only for approved applicants). We introduce a novel cost model for acquiring additional labeled data, designed to more accurately reflect real-world costs such as credit assessment, loan processing, and potential defaults. Our goal is to find optimal fairness audit algorithms that are more cost-effective than random exploration and natural baselines. In our work, we consider two audit settings: a black-box model with no assumptions on the data distribution, and a mixture model, where features and true labels follow a mixture of exponential family distributions. In the black-box setting, we propose a near-optimal auditing algorithm under mild assumptions and show that a natural baseline can be strictly suboptimal. In the mixture model setting, we design a novel algorithm that achieves significantly lower audit cost than the black-box case. Our approach leverages prior work on learning from truncated samples and maximum-a-posteriori oracles, and extends known results on spherical Gaussian mixtures to handle exponential family mixtures, which may be of independent interest. Moreover, our algorithms apply to popular fairness metrics including demographic parity, equal opportunity, and equalized odds. Empirically, we demonstrate strong performance of our algorithms on real-world fair classification datasets like Adult Income and Law School, consistently outperforming natural baselines by around 50% in terms of audit cost.

公平性审计成本优化部分反馈

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。