arXiv:2601.23221cs.LG2026-01被引 1

解决众包标注中的公平性问题,确保不同群体标注结果一致。

Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints

  • 基于人口均等性约束,改进众包标注聚合方法
  • 理论证明标注公平性随人数增长指数收敛
  • 适用于对公平性敏感的数据标注场景

由于获取可靠真实标签通常成本高昂或不可行,众包和噪声标注聚合成为常用方案。然而,主观标注可能放大个体偏见,尤其在敏感特征上引发公平性问题。当前众包聚合的公平性研究仍不充分,缺乏收敛保证,且仅有有限的后处理方法能实现ε-公平性。本文在ε-公平性框架下分析了多数投票与最优贝叶斯聚合的公平性表现。在小规模众包情形下,推导出多数投票公平性差距的上界,该上界依赖于个体标注者的公平性差距。进一步证明,在可解释条件下,聚合共识的公平性差距以指数速度收敛至真实标签水平。由于真实标签本身也可能存在不公平,本文将一种前沿的多分类公平性后处理算法从连续域推广到离散设置,可对任意聚合规则施加严格的人口均等性约束。在合成与真实数据集上的实验验证了该方法的有效性,并支持了理论发现。

原文摘要 · Abstract (English)

As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort. Aggregating subjective labels, though, may amplify individual biases, particularly regarding sensitive features, raising fairness concerns. Nonetheless, fairness in crowdsourced aggregation remains largely unexplored, with no existing convergence guarantees and only limited post-processing approaches for enforcing $\varepsilon$-fairness under demographic parity. We address this gap by analyzing the fairness s of crowdsourced aggregation methods within the $\varepsilon$-fairness framework, for Majority Vote and Optimal Bayesian aggregation. In the small-crowd regime, we derive an upper bound on the fairness gap of Majority Vote in terms of the fairness gaps of the individual annotators. We further show that the fairness gap of the aggregated consensus converges exponentially fast to that of the ground-truth under interpretable conditions. Since ground-truth itself may still be unfair, we generalize a state-of-the-art multiclass fairness post-processing algorithm from the continuous to the discrete setting, which enforces strict demographic parity constraints to any aggregation rule. Experiments on synthetic and real datasets demonstrate the effectiveness of our approach and corroborate the theoretical insights.

众包标注公平性聚合算法人口均等

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。