用微调扩散模型生成均衡数据，缓解图像分类中的群体偏见
Harnessing Diffusion-Generated Synthetic Images for Fair Image Classification
- 通过LoRA和DreamBooth微调扩散模型，精准生成各群体均衡图像
- 在多个基准上优于原始Stable Diffusion，且在严重偏见数据下超越主流去偏方法
- 按组内差异聚类训练，避免单个模型过载，提升生成稳定性
图像分类系统常因训练数据中群体分布不均而继承偏见。例如，在发色分类的面部数据集中，金发常与女性过度关联，强化刻板印象。现有方法利用Stable Diffusion生成平衡数据，但难以保持原数据分布。本文探索多种扩散模型微调技术（如LoRA、DreamBooth），直接从各群体样本中学习，生成更准确的代表性图像。为防止单个DreamBooth模型被组内过高变异淹没，我们采用组内聚类策略，对每个簇训练独立的DreamBooth模型，再用于预训练数据生成，最后在真实数据上微调。实验表明，所提方法平均优于原始Stable Diffusion，性能接近当前最优去偏技术（如Group-DRO），并在数据偏见越严重时表现更优。
原文摘要 · Abstract (English)
Image classification systems often inherit biases from uneven group representation in training data. For example, in face datasets for hair color classification, blond hair may be disproportionately associated with females, reinforcing stereotypes. A recent approach leverages the Stable Diffusion model to generate balanced training data, but these models often struggle to preserve the original data distribution. In this work, we explore multiple diffusion-finetuning techniques, e.g., LoRA and DreamBooth, to generate images that more accurately represent each training group by learning directly from their samples. Additionally, in order to prevent a single DreamBooth model from being overwhelmed by excessive intra-group variations, we explore a technique of clustering images within each group and train a DreamBooth model per cluster. These models are then used to generate group-balanced data for pretraining, followed by fine-tuning on real data. Experiments on multiple benchmarks demonstrate that the studied finetuning approaches outperform vanilla Stable Diffusion on average and achieve results comparable to SOTA debiasing techniques like Group-DRO, while surpassing them as the dataset bias severity increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。