通过调整提示词嵌入,减少图像生成中的性别与种族偏见。
FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models
- 在提示词嵌入空间中用公平主成分分析去除非公平信息。
- 多属性同时去偏,且图像质量损失小。
- 无需重训练,适合各类文生图模型快速部署。
文生图扩散模型(如 Stable Diffusion)虽能生成高质量图像,但常复制社会偏见,尤其体现在性别和种族等人口属性上。本文提出 FairImagen,一种无需重训练或修改模型的后处理去偏框架,通过 CLIP 嵌入进行公平主成分分析,将输入嵌入投影至最小化群体特异性信息的子空间,同时保留语义内容。结合经验性噪声注入与统一跨人口属性投影策略,实现多维度去偏。在性别、种族及交叉属性设置下实验表明,FairImagen 显著提升公平性,仅带来适度的图像质量和提示保真度损失,优于现有后处理方法,提供简单、可扩展、模型无关的公平生成解决方案。
原文摘要 · Abstract (English)
Text-to-image diffusion models, such as Stable Diffusion, have demonstrated remarkable capabilities in generating high-quality and diverse images from natural language prompts. However, recent studies reveal that these models often replicate and amplify societal biases, particularly along demographic attributes like gender and race. In this paper, we introduce FairImagen (https://github.com/fuzihaofzh/FairImagen), a post-hoc debiasing framework that operates on prompt embeddings to mitigate such biases without retraining or modifying the underlying diffusion model. Our method integrates Fair Principal Component Analysis to project CLIP-based input embeddings into a subspace that minimizes group-specific information while preserving semantic content. We further enhance debiasing effectiveness through empirical noise injection and propose a unified cross-demographic projection method that enables simultaneous debiasing across multiple demographic attributes. Extensive experiments across gender, race, and intersectional settings demonstrate that FairImagen significantly improves fairness with a moderate trade-off in image quality and prompt fidelity. Our framework outperforms existing post-hoc methods and offers a simple, scalable, and model-agnostic solution for equitable text-to-image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。