发现并修复文生图模型的文化偏见问题
Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation
- 定位文化敏感神经元，实现精准文化信号激活
- 无需微调模型，推理时即可提升跨语言文化一致性
- 适合关注多语言图像生成公平性的研究者与开发者
多语言文生图模型在视觉真实性和语义对齐方面已取得显著进展，但其输出在不同文化背景下存在差异。由于语言承载文化内涵，多语言提示生成的图像应保持跨语言文化一致性。我们通过全面分析发现，当前模型在多语言提示下常生成文化中立或英语偏倚的结果。对两个代表性模型的分析表明，问题并非缺乏文化知识，而是文化相关表征未能充分激活。为此，我们提出一种探测方法，将文化敏感信号定位到少数固定层中的少量神经元。基于此发现，提出两种互补对齐策略：(1) 推理时文化激活，在不微调主干模型的前提下增强识别出的神经元；(2) 层目标文化增强，仅更新与文化相关的层。在我们的CultureBench数据集上实验表明，该方法在保持图像保真度和多样性的同时，持续优于强基线模型，在文化一致性上表现显著提升。
原文摘要 · Abstract (English)
Multilingual text-to-image (T2I) models have advanced rapidly in terms of visual realism and semantic alignment, and are now widely utilized. Yet outputs vary across cultural contexts: because language carries cultural connotations, images synthesized from multilingual prompts should preserve cross-lingual cultural consistency. We conduct a comprehensive analysis showing that current T2I models often produce culturally neutral or English-biased results under multilingual prompts. Analyses of two representative models indicate that the issue stems not from missing cultural knowledge but from insufficient activation of culture-related representations. We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers. Guided by this finding, we introduce two complementary alignment strategies: (1) inference-time cultural activation that amplifies the identified neurons without backbone fine-tuned; and (2) layer-targeted cultural enhancement that updates only culturally relevant layers. Experiments on our CultureBench demonstrate consistent improvements over strong baselines in cultural consistency while preserving fidelity and diversity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。