arXiv:2503.12271cs.CV2025-03ICCV被引 45

让文生图模型在生成时自我反思,用先前结果改进输出质量。

Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection

  • 引入上下文反思机制,利用已生成图像和文本反馈迭代优化
  • 仅20样本/提示即达GenEval新纪录0.81,超越大模型4.8B参数量表现
  • 适合追求高效高质图像生成的开发者与研究者

当前文生图模型性能提升主要依赖训练时扩容,但计算成本高昂。本文提出反射式扩散变换器(Reflect-DiT),通过引入上下文反思能力实现推理时扩展。该方法使模型在生成过程中,结合先前生成图像与文本反馈,主动修正不足。实验表明,基于SANA-1.0-1.6B模型,Reflect-DiT在GenEval基准上提升0.19分;仅生成20个样本/提示,即达到0.81的最新得分,优于此前使用4.8B参数模型、生成2048个样本的best-of-N方法(原得分为0.80)。此方法显著提升了生成效率与质量。

原文摘要 · Abstract (English)

The predominant approach to advancing text-to-image generation has been training-time scaling, where larger models are trained on more data using greater computational resources. While effective, this approach is computationally expensive, leading to growing interest in inference-time scaling to improve performance. Currently, inference-time scaling for text-to-image diffusion models is largely limited to best-of-N sampling, where multiple images are generated per prompt and a selection model chooses the best output. Inspired by the recent success of reasoning models like DeepSeek-R1 in the language domain, we introduce an alternative to naive best-of-N sampling by equipping text-to-image Diffusion Transformers with in-context reflection capabilities. We propose Reflect-DiT, a method that enables Diffusion Transformers to refine their generations using in-context examples of previously generated images alongside textual feedback describing necessary improvements. Instead of passively relying on random sampling and hoping for a better result in a future generation, Reflect-DiT explicitly tailors its generations to address specific aspects requiring enhancement. Experimental results demonstrate that Reflect-DiT improves performance on the GenEval benchmark (+0.19) using SANA-1.0-1.6B as a base model. Additionally, it achieves a new state-of-the-art score of 0.81 on GenEval while generating only 20 samples per prompt, surpassing the previous best score of 0.80, which was obtained using a significantly larger model (SANA-1.5-4.8B) with 2048 samples under the best-of-N approach.

文生图扩散模型推理优化自我反思

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。