arXiv:2604.25314cs.CV2026-04被引 2

让文生图更准:根据图像区域动态调整噪声,提升复杂提示的生成一致性。

Golden RPG: Confidence-Adaptive Region-Aware Noise for Compositional Text-to-Image Generation

论文配图:Golden RPG: Confidence-Adaptive Region-Aware Noise for Compositional Text-to-Image Generation
图 1 · 摘自论文原文
  • 分区域预测噪声,用FiLM适配器和跨注意力增强局部语义控制
  • 在多区域测试中全面领先,跨区域一致性最高,且保持高质量生成
  • 自适应融合机制,避免简单提示被过度干扰,适合复杂场景生成

组合式文生图要求模型准确响应多个描述不同图像区域的子提示。近期研究表明,扩散模型的起始噪声蕴含重要语义信息:从文本预测出的“黄金”噪声可显著提升提示忠实度。然而,现有方法本质上是全局的——同一网络需将长而多区域的提示压缩为单一文本嵌入,当提示包含空间分离的实体时成为瓶颈。本文提出黄金RPG(Golden RPG),在冻结的NPNet基础上增加两个可训练模块:(i) 每区域的FiLM适配器,按子提示重塑预测噪声;(ii) 注入Swin主干两阶段间的区域交叉注意力层,使不同空间位置能关注不同子提示令牌。为防止区域条件干扰本已准确的生成,进一步设计置信度自适应混合头,动态预测每样本中区域信号应覆盖全局信号的程度。在原始RPG基准(20提示,100样本)及T2I-CompBench四个多区域类别(1,200张图,六种对比方法)上评估,黄金RPG在所有类别中均取得最高跨区域一致性得分,同时在绝对CLIP-Score与CLIP-IQA上达到最强基线水平。成对用户研究显示其偏好度比最强基线高出约67%。适配器含约200万可训练参数,推理额外耗时仅0.6秒(基于SDXL)。

原文摘要 · Abstract (English)

Compositional text-to-image (T2I) generation requires a model to honour multiple sub-prompts that describe distinct image regions. Recent work shows that the \emph{starting noise} of a diffusion model carries significant semantic information: ``golden'' noise predicted from text can substantially raise prompt fidelity. We observe that this noise prediction is, however, fundamentally global: the same network is asked to summarise a long, multi-region prompt with a single text embedding, which becomes the bottleneck whenever the prompt describes scenes with spatially-separated entities. We introduce \textbf{Golden RPG}, a region-aware noise predictor that extends a frozen NPNet with two trainable additions: (i) a per-region \textbf{FiLM adapter} that reshapes the predicted noise according to each sub-prompt; and (ii) a \textbf{Region Cross-Attention} layer injected between two stages of the Swin backbone, allowing different spatial locations to attend to different sub-prompt tokens. To prevent the regional conditioning from degrading samples whose prompts are already easy, we further propose a \textbf{Confidence-Adaptive Blending} head that dynamically predicts, per sample, how strongly the regional signal should override the global signal. We evaluate on the original RPG benchmark (20 prompts, 100 samples) and on four multi-region categories of T2I-CompBench (1{,}200 images, six competing methods). Golden RPG achieves the highest Cross-Region-Coherence score on every category, while matching the strongest baselines on absolute CLIP-Score and CLIP-IQA. A paired user study further shows a $\boldsymbol{\sim}$67\% preference over the strongest baseline. The adapter contains $\sim$2M trainable parameters and adds only $0.6$\,s of inference overhead on top of SDXL.

文生图扩散模型区域控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。