揭示文本生成中物体与属性绑定如何放大偏见,提出量化方法并实现更优去偏。
How Bias Binds: Measuring Hidden Associations for Bias Control in Text-to-Image Compositions
- 提出偏见依附度评分,量化对象-属性绑定对偏见的影响。
- 无需训练的控制框架使组合生成去偏效果提升超10%。
- 揭示去偏需平衡语义关系,适合关注生成公平性的研究者。
文本到图像生成模型常表现出与敏感属性相关的偏见。现有研究多聚焦单一物体提示,缺乏上下文多样性。实际上,提示中每个对象或属性均可贡献偏见,例如“一名戴粉帽的助手”可能隐含女性倾向。被忽视的语义绑定联合效应导致当前去偏方法失效。本文首次探究语义绑定下的偏见表现,证明底层偏见分布会因绑定关系被放大。为此,我们提出偏见依附度评分,量化特定对象-属性绑定激活偏见的程度。进一步构建无需训练的上下文-偏见控制框架,探索标记解耦在去偏语义绑定中的作用。该框架在组合生成任务中实现超过10%的去偏提升。对多种属性-对象绑定的偏见评分及标记解相关分析表明:去偏必须避免破坏关键语义关系。这些发现暴露了当前去偏策略在语义绑定情境下的根本局限,亟需重新评估主流偏见缓解方法。
原文摘要 · Abstract (English)
Text-to-image generative models often exhibit bias related to sensitive attributes. However, current research tends to focus narrowly on single-object prompts with limited contextual diversity. In reality, each object or attribute within a prompt can contribute to bias. For example, the prompt "an assistant wearing a pink hat" may reflect female-inclined biases associated with a pink hat. The neglected joint effects of the semantic binding in the prompts cause significant failures in current debiasing approaches. This work initiates a preliminary investigation on how bias manifests under semantic binding, where contextual associations between objects and attributes influence generative outcomes. We demonstrate that the underlying bias distribution can be amplified based on these associations. Therefore, we introduce a bias adherence score that quantifies how specific object-attribute bindings activate bias. To delve deeper, we develop a training-free context-bias control framework to explore how token decoupling can facilitate the debiasing of semantic bindings. This framework achieves over 10% debiasing improvement in compositional generation tasks. Our analysis of bias scores across various attribute-object bindings and token decorrelation highlights a fundamental challenge: reducing bias without disrupting essential semantic relationships. These findings expose critical limitations in current debiasing approaches when applied to semantically bound contexts, underscoring the need to reassess prevailing bias mitigation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。