通过解析扩散模型内部机制,精准调控生成内容中的性别种族偏见。
Dissecting and Mitigating Diffusion Bias via Mechanistic Interpretability
- 识别模型内部导致偏见的决策特征,实现定向干预。
- 在保持图像质量前提下,有效调节生成内容的偏见水平。
- 揭示了细粒度生成控制机制,推动模型可解释性研究。
扩散模型在生成多样化内容方面表现出色,但其输出常延续社会偏见,如性别与种族歧视,可能加剧现实中的不平等。现有研究多聚焦于生成引导,却忽视了模型内部因果驱动偏见的机制。本文深入分析扩散模型内部过程,发现嵌入架构中的特定决策机制——偏见特征。通过直接操控这些特征,方法可精确隔离并调整导致偏见生成的成分,实现对生成内容偏见程度的细粒度控制。在无条件与条件扩散模型上,针对多种社会偏见属性的实验表明,该方法能有效管理生成分布,同时保持高质量输出。此外,还剖析了所发现的模型机制,揭示不同内在特征控制生成的细微层面,为扩散模型的机制可解释性研究提供新视角。
原文摘要 · Abstract (English)
Diffusion models have demonstrated impressive capabilities in synthesizing diverse content. However, despite their high-quality outputs, these models often perpetuate social biases, including those related to gender and race. These biases can potentially contribute to harmful real-world consequences, reinforcing stereotypes and exacerbating inequalities in various social contexts. While existing research on diffusion bias mitigation has predominantly focused on guiding content generation, it often neglects the intrinsic mechanisms within diffusion models that causally drive biased outputs. In this paper, we investigate the internal processes of diffusion models, identifying specific decision-making mechanisms, termed bias features, embedded within the model architecture. By directly manipulating these features, our method precisely isolates and adjusts the elements responsible for bias generation, permitting granular control over the bias levels in the generated content. Through experiments on both unconditional and conditional diffusion models across various social bias attributes, we demonstrate our method's efficacy in managing generation distribution while preserving image quality. We also dissect the discovered model mechanism, revealing different intrinsic features controlling fine-grained aspects of generation, boosting further research on mechanistic interpretability of diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。