定位扩散模型隐性决策层，精准干预提升生成公平性。
Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models
- 通过探测法识别概念区分度最高的注意力层
- 仅干预少数注意力层即可显著减少生成偏差
- 适合研究生成模型可解释性与可控生成的学者
文本到图像扩散模型具备强大生成能力，但其内部运作机制仍不透明，尤其在提示信息不完整时需自主决定未明确描述的细节。本文提出假设：这些隐性决策并非全局分布，而是计算上集中于模型特定层级。现有定位方法多针对显式提示干预，而我们发现其与隐性决策存在差异。因此引入基于探测的定位技术，识别属性区分度最高的层。结果表明，模糊概念的解析主要由自注意力层主导，是干预的最佳位置。据此提出ICM（隐性选择修改）方法，对少量特定层进行精准干预。大量实验验证，该方法在去偏性能上优于现有最优方法，且显著降低粗略干预带来的伪影。代码已开源。
原文摘要 · Abstract (English)
Text-to-image diffusion models exhibit remarkable generative capabilities, yet their internal operations remain opaque, particularly when handling prompts that are not fully descriptive. In such scenarios, models must make implicit decisions to generate details not explicitly specified in the text. This work investigates the hypothesis that this decision-making process is not diffuse but is computationally localized within the model's architecture. While existing localization techniques focus on prompt-related interventions, we notice that such explicit conditioning may differ from implicit decisions. Therefore, we introduce a probing-based localization technique to identify the layers with the highest attribute separability for concepts. Our findings indicate that the resolution of ambiguous concepts is governed principally by self-attention layers, identifying them as the most effective point for intervention. Based on this discovery, we propose ICM (Implicit Choice-Modification) - a precise steering method that applies targeted interventions to a small subset of layers. Extensive experiments confirm that intervening on these specific self-attention layers yields superior debiasing performance compared to existing state-of-the-art methods, minimizing artifacts common to less precise approaches. The code is available at https://github.com/kzaleskaa/icm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。