用遗传算法优化样本权重,能更好平衡公平性与预测性能。
Evolved Sample Weights for Bias Mitigation: Effectiveness Depends on the Fairness Objective
- 用遗传算法进化样本权重,自动调整数据点重要性
- 在11个数据集上,进化权重在多数情况下更优
- 对不同公平目标效果差异大,尤其适合追求群体平等
基于真实世界数据训练的机器学习模型可能产生歧视性预测,影响边缘群体。重加权通过为每个训练样本分配权重来缓解此类偏见,但有时会牺牲预测准确率。本文比较了三种生成权重的方法:(1)使用遗传算法(GA)演化权重,(2)仅根据数据集特征计算权重,(3)所有样本赋予相等权重。在11个公开数据集(包括两个医疗数据集)上,采用配对预测与公平性指标评估模型表现。使用准确率、AUC、群体公平性(demographic parity)和子组假阴性公平性作为评估标准。实验表明,进化权重可实现更优的公平性与性能权衡,但其增益程度强烈依赖于所选公平性目标。当以群体公平性为目标时,进化权重在最多数据集上优于其他策略,且不依赖性能目标选择。
原文摘要 · Abstract (English)
Machine learning models trained on real-world data may inadvertently make biased predictions that negatively impact marginalized communities. Reweighting, which assigns a weight to each data point used during model training, can mitigate such bias, though sometimes at the cost of predictive accuracy. In this paper, we investigated this trade-off by comparing three methods for generating these weights: (1) evolving them using a Genetic Algorithm (GA), (2) computing them using only dataset characteristics, and (3) assigning equal weights to all data points. Model performance under each strategy was evaluated using paired predictive and fairness metrics. We used two predictive metrics (accuracy and area under the Receiver Operating Characteristic curve) and two fairness metrics (demographic parity and subgroup false negative fairness). By conducting experiments on eleven publicly available datasets (including two medical datasets), we show that evolved sample weights can produce models that achieve better trade-offs between fairness and predictive performance than alternative weighting methods. However, the magnitude of these benefits depends strongly on the choice of fairness objective. Our experiments reveal that the evolved weights were most effective when optimizing for demographic parity-independent of choice of the performance objective-yielding better performance than other weighting strategies on the largest number of datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。