揭示采样偏差的根源，提出两种新分类并给出公平性评估改进建议。
On the Origins of Sampling Bias: Implications on Fairness Measurement and Mitigation
- 区分样本量偏差与代表性不足偏差两类采样偏差。
- 实验表明不同群体受采样偏差影响程度不同，可能加剧歧视。
- 为模型训练者提供可操作的公平性测量改进方案。
准确衡量歧视是评估机器学习模型公平性的关键。测量中的偏差会导致现有差异被放大或低估。现有研究假设不同群体（如男女、黑白人群）遭受的机器学习偏差均等。但若偏差在不同群体中不均等产生，可能加剧对特定子群体的歧视。特别是，采样偏差在文献中被模糊使用，指代因采样过程引发的偏差。本文通过引入明确的采样偏差变体——样本量偏差（SSB）和代表性不足偏差（URB），在基准数据集上使用主流学习算法进行大量实验，揭示了多种模型训练场景下的关键发现，并最终提出可供实践者采纳的行动建议。
原文摘要 · Abstract (English)
Accurately measuring discrimination is crucial to faithfully assessing fairness of trained machine learning (ML) models. Any bias in measuring discrimination leads to either amplification or underestimation of the existing disparity. Several sources of bias exist and it is assumed that bias resulting from machine learning is born equally by different groups (e.g. females vs males, whites vs blacks, etc.). If, however, bias is born differently by different groups, it may exacerbate discrimination against specific sub-populations. Sampling bias, in particular, is inconsistently used in the literature to describe bias due to the sampling procedure. In this paper, we attempt to disambiguate this term by introducing clearly defined variants of sampling bias, namely, sample size bias (SSB) and underrepresentation bias (URB). Through an extensive set of experiments on benchmark datasets and using mainstream learning algorithms, we expose relevant observations in several model training scenarios. The observations are finally framed as actionable recommendations for practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。