通过调整生成过程消除文生图模型偏见,不降质量且零计算开销。
Inference Time Debiasing Concepts in Diffusion Models
- 仅修改推理流程,避开包含偏见的潜在空间区域。
- 人工评估1200张图,显著降低性别、种族、年龄偏见。
- 适合希望快速部署去偏方案的开发者与应用方。
我们提出DeCoDi,一种针对基于扩散模型的文生图模型的去偏方法,仅改变推理过程,不影响图像质量且计算开销极低,可应用于任意扩散模型。DeCoDi通过调整扩散过程,避开包含偏见概念的潜在空间区域。相较于需要复杂干预的现有方法,本方法仅在推理阶段操作,更易被广泛采用。我们在护士、消防员和首席执行官等概念上对性别、种族和年龄偏见进行去偏测试。两名独立评估者手动检查了1200张生成图像,结果表明该方法有效缓解了三类偏见。自动评估使用GPT4o,其结果与人工评估无显著统计差异。评估显示评价者间一致性高,覆盖更多受保护属性,证明该方法有望显著提升扩散模型生成图像的多样性。
原文摘要 · Abstract (English)
We propose DeCoDi, a debiasing procedure for text-to-image diffusion-based models that changes the inference procedure, does not significantly change image quality, has negligible compute overhead, and can be applied in any diffusion-based image generation model. DeCoDi changes the diffusion process to avoid latent dimension regions of biased concepts. While most deep learning debiasing methods require complex or compute-intensive interventions, our method is designed to change only the inference procedure. Therefore, it is more accessible to a wide range of practitioners. We show the effectiveness of the method by debiasing for gender, ethnicity, and age for the concepts of nurse, firefighter, and CEO. Two distinct human evaluators manually inspect 1,200 generated images. Their evaluation results provide evidence that our method is effective in mitigating biases based on gender, ethnicity, and age. We also show that an automatic bias evaluation performed by the GPT4o is not significantly statistically distinct from a human evaluation. Our evaluation shows promising results, with reliable levels of agreement between evaluators and more coverage of protected attributes. Our method has the potential to significantly improve the diversity of images it generates by diffusion-based text-to-image generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。