arXiv:2603.20828cs.CV2026-03被引 1

让扩散模型理解隐含知识,生成更符合常识的图像。

EruDiff: Refactoring Knowledge in Diffusion Models for Advanced Text-to-Image Synthesis

  • 通过知识分布匹配对齐隐式与显式提示的语义结构。
  • 在Science-T2I和WISE基准上显著提升生成准确性。
  • 适合需要常识推理的图像生成研究者使用。

文本到图像扩散模型在生成基于明确文本提示的图像方面表现出色,但在处理需深层世界知识(如自然科学、文化常识)的隐含提示时存在明显缺陷,导致生成结果违背事实。本文发现该问题源于底层知识结构的错位,表现为隐式提示的组织混乱。为此,提出EruDiff,通过扩散知识分布匹配(DK-DM)将难以处理的隐式提示知识分布与定义良好的显式锚点对齐,并采用仅负样本强化学习(NO-RL)策略精细修正显式提示中的固有偏差。大量实验证明,该方法显著提升了FLUX和Qwen-Image等主流模型在Science-T2I(科学知识)与WISE(世界知识)基准上的表现,验证了其有效性与通用性。代码已开源。

原文摘要 · Abstract (English)

Text-to-image diffusion models have achieved remarkable fidelity in synthesizing images from explicit text prompts, yet exhibit a critical deficiency in processing implicit prompts that require deep-level world knowledge, ranging from natural sciences to cultural commonsense, resulting in counter-factual synthesis. This paper traces the root of this limitation to a fundamental dislocation of the underlying knowledge structures, manifesting as a chaotic organization of implicit prompts compared to their explicit counterparts. In this paper, we propose EruDiff, which aims to refactor the knowledge within diffusion models. Specifically, we develop the Diffusion Knowledge Distribution Matching (DK-DM) to register the knowledge distribution of intractable implicit prompts with that of well-defined explicit anchors. Furthermore, to rectify the inherent biases in explicit prompt rendering, we employ the Negative-Only Reinforcement Learning (NO-RL) strategy for fine-grained correction. Rigorous empirical evaluations demonstrate that our method significantly enhances the performance of leading diffusion models, including FLUX and Qwen-Image, across both the scientific knowledge benchmark (i.e., Science-T2I) and the world knowledge benchmark (i.e., WISE), underscoring the effectiveness and generalizability. Our code is available at https://github.com/xiefan-guo/erudiff.

图像生成扩散模型常识推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。