arXiv:2606.24888cs.CV2026-06被引 2

提出DiffusionBench,统一评估扩散Transformer在图像和文本生成上的表现。

DiffusionBench: On Holistic Evaluation of Diffusion Transformers

论文配图:DiffusionBench: On Holistic Evaluation of Diffusion Transformers
图 1 · 摘自论文原文
  • 构建NanoGen框架,仅改12行配置即可训练文本到图像模型
  • 发现图像生成与文本生成排名相关性低,跨任务评估必要
  • 推荐用DiffusionBench替代单一ImageNet评估,更全面反映进展

扩散变压器(DiT)在图像生成研究中趋于统一评估标准:在ImageNet上进行类别条件生成。尽管方法在FID等指标上持续提升,但其是否真正反映生成建模的进步已存疑。文本到图像(T2I)生成常被认为训练和评估成本过高而被跳过。我们指出这一认知已过时。本文提出NanoGen,一个统一的DiT训练与评估框架,在ImageNet上达到顶尖基线性能,并通过12行配置变更即可训练出具有竞争力的T2I模型。目前支持RAE、VAE、像素空间和MeanFlow四种扩散方法,覆盖ImageNet与T2I场景。在相同计算开销下,训练T2I与ImageNet相当。基于21个潜在扩散模型的实验表明,方法在ImageNet与T2I上的排名相关性弱(皮尔逊相关系数介于-0.377至-0.580之间),说明优化类条件生成未必提升文本生成性能。为此,我们整合两者结果,构建了涵盖两类任务的完整基准——DiffusionBench。建议以DiffusionBench替代仅报告ImageNet结果,更能体现模型的广泛进步。

原文摘要 · Abstract (English)

Diffusion transformer (DiT) research on image generation has converged to a single evaluation setup: class-conditional generation on ImageNet. While methods improve the FID and related metrics, it is increasingly unclear whether they reflect real progress in generative modeling. The natural alternative, i.e., text-to-image (T2I) generation, is perceived as too costly or inconvenient to train and evaluate and is often skipped. We argue that this perception no longer holds. We introduce NanoGen, a unified DiT training and evaluation framework. NanoGen matches state-of-the-art DiT baselines on ImageNet and, with 12 lines of configuration change, also trains competitive text-to-image models. It currently supports RAE, VAE, pixel-space, and MeanFlow diffusion methods under both ImageNet and T2I setups. Under NanoGen, training T2I requires comparable compute to ImageNet. After training 21 latent diffusion models with NanoGen, we observe that method ranking shows no strong correlation between ImageNet and T2I generation: Pearson correlation is between -0.377 and -0.580 across three metrics. This suggests that a method which improves class-conditional ImageNet FID may show no corresponding improvement on T2I, clearly indicating the necessity of evaluating DiTs on both tasks. To this end, we summarize ImageNet and text-to-image results, which yields DiffusionBench, a holistic benchmark for DiT research. We recommend reporting DiffusionBench in place of ImageNet alone: methods that improve DiffusionBench are more likely to reflect broader progress.

扩散模型图像生成文本生成评估基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。