arXiv:2411.01685cs.LGcs.CY2024-11

提出新方法校准匹配分数,消除模型在不同群体间的隐性偏差。

Reducing Biases in Record Matching Through Scores Calibration

  • 用全阈值积分定义分数偏差,突破固定阈值评估局限。
  • 在多个基准上验证,现有深度匹配器虽表面公平,实则存在显著分数偏差。
  • 无需重训练,仅需校准集即可通过最优传输减少偏差,适合实际部署场景。

记录匹配模型通常输出一个实数值匹配分数,后续通过阈值设定、排序或人工审核使用。尽管公平性评估多基于固定阈值下的二元决策,但此类方法可能忽略整个分数分布中的系统性差异,且结论随阈值变化。本文引入一种不依赖阈值的分数偏差概念,将标准群公平性准则(人口均等性DP、平等机会EO、平等几率EOD)从二元输出扩展至分数函数,通过在所有阈值上积分群体间度量差距实现。实验表明,多个前沿深度匹配器在常用阈值下看似公平,却在分数分布上存在显著偏差。为缓解此问题而不需重训练,本文提出两种模型无关的后处理方法:Calib通过分位数最优传输映射,将少数/多数群体分数分布对齐至共同的Wasserstein质心,具有有限样本下残余偏差与分数失真的保证;C-Calib进一步将该思想拓展至标签相关公平性(EO/EOD),条件性地进行质心对齐,并分析其保证对样本量与标签估计误差的依赖。在标准记录匹配基准和多种神经匹配器上的实验表明,Calib与C-Calib可显著降低分数偏差,同时保持精度损失极小。

原文摘要 · Abstract (English)

Record matching models typically output a real-valued matching score that is later consumed through thresholding, ranking, or human review. While fairness in record matching has mostly been assessed using binary decisions at a fixed threshold, such evaluations can miss systematic disparities in the entire score distribution and can yield conclusions that change with the chosen threshold. We introduce a threshold-independent notion of score bias that extends standard group-fairness criteria-demographic parity (DP), equal opportunity (EO), and equalized odds (EOD)-from binary outputs to score functions by integrating group-wise metric gaps over all thresholds. Using this metric, we empirically show that several state-of-the-art deep matchers can exhibit substantial score bias even when appearing fair at commonly used thresholds. To mitigate these disparities without retraining the underlying matcher, we propose two model-agnostic post-processing methods that only require score evaluations on an (unlabeled) calibration set. Calib targets DP by aligning minority/majority score distributions to a common Wasserstein barycenter via a quantile-based optimal-transport map, with finite-sample guarantees on both residual DP bias and score distortion. C-Calib extends this idea to label-dependent notions (EO/EOD) by performing barycenter alignment conditionally on an estimated label, and we characterize how its guarantees depend on both sample size and label-estimation error. Experiments on standard record-matching benchmarks and multiple neural matchers confirm that Calib and C-Calib substantially reduce score bias with minimal loss in accuracy.

记录匹配公平性偏差校准后处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。