arXiv:2605.19532cs.CVcs.LG2026-05

通过核心词注意力选种子,提升文生图模型一致性与质量

Boosting Text-to-Image Diffusion Models via Core Token Attention-Based Seed Selection

论文配图:Boosting Text-to-Image Diffusion Models via Core Token Attention-Based Seed Selection
图 1 · 摘自论文原文
  • 基于提示词核心词在去噪初期的注意力动态筛选种子
  • 无需微调,在Stable Diffusion上显著提升图像质量和对齐度
  • 适合需要稳定生成结果的文生图应用,可无缝接入现有流程

文生图扩散模型虽能生成高质量图像,但结果对随机种子极为敏感:不同种子常导致图像质量与提示词对齐度差异显著。本文重新审视这一‘种子效应’,发现提示词核心词(承载内容的词汇)在前几轮去噪过程中的注意力动态,能有效预测最终生成质量。基于此,提出无需训练、即插即用的注意力种子选择方法(ABSS),通过在去噪过程中对核心词的跨注意力进行评分,对候选种子排序并保留前k个进行完整生成,其余直接舍弃,不依赖固定阈值。该方法仅在推理阶段运行,可作为轻量级预筛选模块,集成至现有种子优化流程中,实现额外增益。在三个基准测试中,实验表明ABSS能持续提升Stable Diffusion系列模型的文本-图像对齐度与视觉质量,人类偏好与对齐指标均验证了其有效性。

原文摘要 · Abstract (English)

Text-to-image diffusion models can synthesize high-quality images, yet the outcome is notoriously sensitive to the random seed: different initial seeds often yield large variations in image quality and prompt-image alignment. We revisit this "seed effect" and show that attention dynamics over prompt core tokens, the content-bearing words, measured during the first few denoising steps, strongly predict final generation quality. Building on this observation, we introduce Attention-Based Seed Selection (ABSS), a training-free, plug-and-play method that ranks seeds for a given prompt by leveraging cross-attention to core tokens during the denoising process. ABSS requires no finetuning and does not alter the initial noise; it scores and ranks all candidate seeds, keeps only the top-k for full generation, and discards the rest, without relying on a fixed accept/reject threshold. Operating purely at inference time, ABSS can serve as a lightweight pre-selection add-on for existing seed-optimization pipelines, enabling additional gains. Across three benchmarks, extensive experiments show that ABSS enables consistent improvements in text-image alignment and visual quality for Stable Diffusion variants, as corroborated by human preference and alignment metrics.

文生图扩散模型种子优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。