用概念图对齐降低文生图模型偏见，生成更公平且连贯的图像。
Efficient bias mitigation in T2I diffusion models using Concept Graphs
- 通过构建概念图对齐文本编码器与去噪器内部语义结构
- 偏见减少30%，图像质量提升ΔFID=11.4，语义混乱减少88%
- 适合关注模型公平性与生成一致性的研究者
文生图扩散模型常继承训练数据中的有害偏见。现有缓解方法多仅作用于文本编码器或推理时引导，常导致生成结果语义不连贯。为此，我们提出基于概念图对齐的CO-ALIGN（概念本体对齐）方法，直接在模型内部概念本体层面进行干预。通过对齐文本编码器与去噪器中的概念，CO-ALIGN在保持生成完整性的同时显著降低偏见。我们在三种范式下验证：仅文本编码器、仅去噪器、联合文本-去噪器本体对齐。结果表明，相比最先进方法，其公平性提升30%，ΔFID=11.4（图像质量），图像保真度提高2.8%，语义不连贯输出减少88%。此外，我们发现对齐更优的内部本体可增强多种概念删减技术的鲁棒性。
原文摘要 · Abstract (English)
Text-to-Image diffusion models often propagate harmful bias inherited from the training data. Existing bias mitigation techniques typically intervene only at the text encoder or provide inference-time guidance, often leading to generations that collapse into semantically incoherent outputs. To address these limitations, we introduce CO-ALIGN (Concept Ontology Alignment), a novel bias mitigation approach based on concept-graph alignment that operates on the model's internal concept ontology. By aligning concepts within the text encoder and denoiser, CO-ALIGN achieves substantial bias reduction while preserving generative integrity. We demonstrate the effectiveness of concept-graph alignment across three paradigms: text-encoders, denoisers and joint text-denoiser ontology alignment. CO-ALIGN outperforms the state of the art, improving fairness by $30\%$, $ΔFID=11.4$ in image quality, $2.8\%$ in image fidelity, all while reducing semantically incoherent outputs by $88\%$. Beyond bias mitigation, we show that CO-ALIGN benefits other downstream tasks as well. In particular, our experiments demonstrate that better-aligned internal ontologies enhance concept unlearning robustness across multiple unlearning techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。