解决扩散模型生成同形异义词时的歧义混淆问题
Un-Doubling Diffusion: LLM-guided Disambiguation of Homonym Duplication
- 用提示词扩展方法减少同形异义词的重复生成
- 发现英语翻译引入的偏见会加剧歧义现象
- 提出自动评估框架并开源代码，适合多语言生成研究者
同形异义词（拼写相同但意义不同）对生成模型构成挑战。当提示中出现同形异义词时，扩散模型可能同时生成该词的多个含义，称为同形异义词重复。这一问题因英语中心偏见而加剧，即在文本到图像生成流程前需经历额外翻译步骤，导致原文非同形异义词在译为英语后变为同形异义词并丢失原意。本文提出一种测量重复率的方法，并通过视觉-语言模型（VLM）自动评估与人工评估相结合的方式，对多种扩散模型进行评测。此外，研究发现提示词扩展可有效缓解同形异义词重复问题，包括由英语中心偏见引发的重复。相关自动评估代码已公开。
原文摘要 · Abstract (English)
Homonyms are words with identical spelling but distinct meanings, which pose challenges for many generative models. When a homonym appears in a prompt, diffusion models may generate multiple senses of the word simultaneously, which is known as homonym duplication. This issue is further complicated by an Anglocentric bias, which includes an additional translation step before the text-to-image model pipeline. As a result, even words that are not homonymous in the original language may become homonyms and lose their meaning after translation into English. In this paper, we introduce a method for measuring duplication rates and conduct evaluations of different diffusion models using both automatic evaluation utilizing Vision-Language Models (VLM) and human evaluation. Additionally, we investigate methods to mitigate the homonym duplication problem through prompt expansion, demonstrating that this approach also effectively reduces duplication related to Anglocentric bias. The code for the automatic evaluation pipeline is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。