用少量数据微调模型参数,高效降低医疗AI偏见同时保持诊断准确
SWiFT: Soft-Mask Weight Fine-tuning for Bias Mitigation
- 通过分析参数对偏见与性能的贡献,分步调整梯度实现精准去偏
- 仅需少量外部数据和数轮微调,就能在多个皮肤科和胸片数据集上显著降偏
- 适合医疗AI研发者,尤其关注公平性又无原始训练数据的场景
近期研究表明,机器学习模型在真实场景中可能表现出偏见,给医疗等伦理敏感领域带来挑战。这种偏见会损害模型公平性与泛化能力,甚至加剧社会歧视。现有去偏方法通常需要原始训练数据并进行大量重训练,且在公平性与判别性能间存在权衡。为此,我们提出软掩码权重微调(SWiFT)框架,仅需少量外部数据和数轮微调即可高效提升公平性并保留判别性能。其核心思想是先识别模型参数对偏见与预测性能的相对贡献,再通过两阶段微调,依据贡献度赋予不同梯度更新。在四个皮肤病数据集和两个胸部X光数据集上,针对性别、肤色、年龄三个敏感属性的实验表明,SWiFT能持续降低模型偏见,并在常见公平性与准确性指标下实现媲美或优于当前最优水平的诊断准确率。特别地,其在多个分布外(OOD)数据集上表现更优,证明了更强的泛化能力。
原文摘要 · Abstract (English)
Recent studies have shown that Machine Learning (ML) models can exhibit bias in real-world scenarios, posing significant challenges in ethically sensitive domains such as healthcare. Such bias can negatively affect model fairness, model generalization abilities and further risks amplifying social discrimination. There is a need to remove biases from trained models. Existing debiasing approaches often necessitate access to original training data and need extensive model retraining; they also typically exhibit trade-offs between model fairness and discriminative performance. To address these challenges, we propose Soft-Mask Weight Fine-Tuning (SWiFT), a debiasing framework that efficiently improves fairness while preserving discriminative performance with much less debiasing costs. Notably, SWiFT requires only a small external dataset and only a few epochs of model fine-tuning. The idea behind SWiFT is to first find the relative, and yet distinct, contributions of model parameters to both bias and predictive performance. Then, a two-step fine-tuning process updates each parameter with different gradient flows defined by its contribution. Extensive experiments with three bias sensitive attributes (gender, skin tone, and age) across four dermatological and two chest X-ray datasets demonstrate that SWiFT can consistently reduce model bias while achieving competitive or even superior diagnostic accuracy under common fairness and accuracy metrics, compared to the state-of-the-art. Specifically, we demonstrate improved model generalization ability as evidenced by superior performance on several out-of-distribution (OOD) datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。