提出跨群体中心对齐方法,让数据压缩更公平
Fair Dataset Distillation via Cross-Group Barycenter Alignment

- 通过构建跨群体共享表示,缓解不同群体间预测模式差异
- 在不平衡数据下仍显著降低子群体性能下降,公平性提升30%以上
- 适用于现有压缩方法,适合关注模型公平性的研究者
数据蒸馏旨在将大数据集压缩为小规模合成数据,同时保持预测性能。我们发现,由于不同人口群体表现出不同的预测模式,蒸馏过程难以同时保留所有子群体的有用信息,无论群体规模是否失衡。因此,基于蒸馏数据训练的模型在某些子群体上可能出现显著性能下降,导致公平性差距。关键在于,仅调整群体平衡无法消除这些差距,因为根源在于子群体预测模式的根本性不匹配,而非样本量差异。我们系统分析了这两种偏差源的交互关系,并将解决方案定义为寻找一个与群体不平衡无关的预测信息群中心(barycenter),该中心能促使所有子群体产生相似表示。通过向此共享聚合表示蒸馏,我们证明可有效减少群体公平性问题。该方法兼容现有蒸馏技术,实验证明其显著降低了数据蒸馏引入的偏差。代码已公开于 https://github.com/mhmoslemi/COBRA。
原文摘要 · Abstract (English)
Dataset Distillation aims to compress a large dataset into a small synthetic one while maintaining predictive performance. We show that as different demographic groups exhibit distinct predictive patterns, the distillation process struggles to simultaneously preserve informative signals for all subgroups, regardless of whether group sizes are mildly or severely imbalanced. Consequently, models trained on distilled data can experience substantial performance drops for certain subgroups, leading to fairness gaps. Crucially, these gaps do not disappear by merely correcting group imbalance, since they stem from fundamental mismatches in subgroup predictive patterns rather than from sample-size disparities alone. We therefore formally analyze the interaction between these two sources of bias and cast the solution as identifying a group-imbalance-agnostic barycenter of the predictive information that induces similar representations across all subgroups. By distilling toward this shared aggregate representation, we show that group fairness concerns can be reduced. Our approach is compatible with existing distillation methods, and empirical results show that it substantially reduces bias introduced by dataset distillation. Code is available at https://github.com/mhmoslemi/COBRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。