让AI画画更准：通过推理时调整噪声，精准对齐文字描述的空间位置。
InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Models
- 在生成过程中动态调整噪声，利用注意力图引导物体位置。
- 在VISOR和T2I-CompBench上超越现有方法，接近微调效果。
- 无需训练、即插即用，适配任意扩散模型使用。
文本到图像（T2I）扩散模型虽能生成高质量图像，但常无法准确捕捉文本提示中的空间关系。这一局限源于训练数据缺乏细粒度空间监督，以及文本嵌入难以编码空间语义。本文提出InfSplign，一种无需训练的推理时方法，在每个去噪步骤中通过复合损失调整噪声，利用骨干解码器提取的多层级交叉注意力图，强制实现物体精准定位与均衡存在。该方法轻量、可即插即用，兼容任意扩散模型。在VISOR和T2I-CompBench上的全面评估表明，InfSplign达到当前最佳性能（据我们所知），显著优于现有最先进推理时方法，甚至超过微调基线。代码已开源。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models generate high-quality images but often fail to capture the spatial relations specified in text prompts. This limitation can be traced to two factors: lack of fine-grained spatial supervision in training data and inability of text embeddings to encode spatial semantics. We introduce InfSplign, a training-free inference-time method that improves spatial alignment by adjusting the noise through a compound loss in every denoising step. Proposed loss leverages different levels of cross-attention maps extracted from the backbone decoder to enforce accurate object placement and a balanced object presence during sampling. The method is lightweight, plug-and-play, and compatible with any diffusion backbone. Our comprehensive evaluations on VISOR and T2I-CompBench show that InfSplign establishes a new state-of-the-art (to the best of our knowledge), achieving substantial performance gains over the strongest existing inference-time baselines and even outperforming the fine-tuning-based methods. Codebase is available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。