arXiv:2411.19623cs.CVcs.AI2024-11NeurIPS被引 3

让数据蒸馏更公平,避免模型偏见放大少数群体歧视

FairDD: Fair Dataset Distillation

  • 通过同步匹配各群体数据,防止合成数据集中少数群体被忽略
  • 在保持分类准确率的同时,显著降低对性别、种族等属性的不公平性
  • 适用于多种现有蒸馏方法,无需修改原模型架构

将大规模数据集压缩为小型合成数据集在图像分类中展现出巨大潜力。然而,以往研究忽略了图像识别中的关键问题:确保在压缩数据集上训练的模型不对受保护属性(如性别、种族)产生偏见。我们发现,数据蒸馏不仅未能缓解原始数据集中的不公平现象,反而因数据量减小使少数群体的偏见进一步加剧。为此,我们提出一种新型公平数据蒸馏框架 FairDD,可无缝集成到多种基于匹配的数据蒸馏方法中,无需修改原有架构。其核心在于同步将合成数据与原始数据中按受保护属性划分的子群体进行匹配,而非像传统方法那样整体对齐,主导于多数群体的分布。这种同步匹配机制有效防止合成数据向多数群体坍缩,促进对所有受保护属性群体的均衡生成。结果表明,FairDD能有效正则化原有蒸馏方法,增强对少数群体的生成公平性,同时保持目标属性的分类精度。理论分析与大量实验验证显示,相比原始蒸馏方法,FairDD在公平性上取得显著提升,且在公平性与准确率之间实现良好权衡。该方法在分布匹配与梯度匹配类蒸馏方法中均表现出一致性优势,具备高度通用性。代码已公开于 https://github.com/zqhang/FairDD。

原文摘要 · Abstract (English)

Condensing large datasets into smaller synthetic counterparts has demonstrated its promise for image classification. However, previous research has overlooked a crucial concern in image recognition: ensuring that models trained on condensed datasets are unbiased towards protected attributes (PA), such as gender and race. Our investigation reveals that dataset distillation fails to alleviate the unfairness towards minority groups within original datasets. Moreover, this bias typically worsens in the condensed datasets due to their smaller size. To bridge the research gap, we propose a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures. The key innovation of FairDD lies in synchronously matching synthetic datasets to PA-wise groups of original datasets, rather than indiscriminate alignment to the whole distributions in vanilla DDs, dominated by majority groups. This synchronized matching allows synthetic datasets to avoid collapsing into majority groups and bootstrap their balanced generation to all PA groups. Consequently, FairDD could effectively regularize vanilla DDs to favor biased generation toward minority groups while maintaining the accuracy of target attributes. Theoretical analyses and extensive experimental evaluations demonstrate that FairDD significantly improves fairness compared to vanilla DDs, with a promising trade-off between fairness and accuracy. Its consistent superiority across diverse DDs, spanning Distribution and Gradient Matching, establishes it as a versatile FDD approach. Code is available at https://github.com/zqhang/FairDD.

数据蒸馏公平性图像分类偏见缓解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。