arXiv:2603.20741cs.CV2026-03中稿 · CVPR被引 1

通过跨时间步自校准,提升文本图像生成的精准对齐。

CTCal: Rethinking Text-to-Image Diffusion Models via Cross-Timestep Self-Calibration

  • 利用低噪声时段的图文对齐信息,校准高噪声时段的表征学习。
  • 在T2I-Compbench++和GenEval上显著提升生成质量与对齐度。
  • 适用于扩散模型与流模型,可无缝集成到现有架构中。

基于扩散模型的文本到图像合成虽有进展,但实现文本提示与生成图像间的精确对齐仍是难题。我们发现,这主要源于传统扩散损失仅提供隐式监督,难以建模细粒度图文对应关系。本文提出跨时间步自校准(CTCal),基于观察:随着扩散过程推进、时间步增大,图文对齐难度逐渐上升。CTCal利用低时间步(噪声较少)时形成的可靠图文对齐(即交叉注意力图)来校准高时间步(噪声较多)的表示学习,从而在训练中引入显式监督。我们进一步设计了时间步感知的自适应加权机制,实现CTCal与扩散损失的和谐融合。该方法具有模型无关性,可无缝集成至现有文本到图像扩散模型(如SD 2.1)及流模型(如SD 3)。在T2I-Compbench++和GenEval基准上的大量实验验证了其有效性与通用性。代码已开源:https://github.com/xiefan-guo/ctcal。

原文摘要 · Abstract (English)

Recent advancements in text-to-image synthesis have been largely propelled by diffusion-based models, yet achieving precise alignment between text prompts and generated images remains a persistent challenge. We find that this difficulty arises primarily from the limitations of conventional diffusion loss, which provides only implicit supervision for modeling fine-grained text-image correspondence. In this paper, we introduce Cross-Timestep Self-Calibration (CTCal), founded on the supporting observation that establishing accurate text-image alignment within diffusion models becomes progressively more difficult as the timestep increases. CTCal leverages the reliable text-image alignment (i.e., cross-attention maps) formed at smaller timesteps with less noise to calibrate the representation learning at larger timesteps with more noise, thereby providing explicit supervision during training. We further propose a timestep-aware adaptive weighting to achieve a harmonious integration of CTCal and diffusion loss. CTCal is model-agnostic and can be seamlessly integrated into existing text-to-image diffusion models, encompassing both diffusion-based (e.g., SD 2.1) and flow-based approaches (e.g., SD 3). Extensive experiments on T2I-Compbench++ and GenEval benchmarks demonstrate the effectiveness and generalizability of the proposed CTCal. Our code is available at https://github.com/xiefan-guo/ctcal.

文本生成扩散模型对齐优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。