用扩散模型联合生成图像和语义特征，提升生成质量与效率
Boosting Generative Image Modeling via Joint Image-Feature Synthesis
- 用扩散模型同时建模图像潜空间和语义特征
- 在无条件/有条件设置下均显著提升图像质量和训练速度
- 无需复杂蒸馏，可直接用于语义引导生成
潜在扩散模型（LDMs）主导高质量图像生成，但表征学习与生成建模的融合仍是挑战。本文提出一种新框架，通过扩散模型联合建模来自变分自编码器的低层图像潜变量和来自预训练自监督编码器（如DINO）的高层语义特征。该潜在-语义扩散方法从纯噪声中生成一致的图像-特征对，显著提升生成质量与训练效率，且仅需对标准Diffusion Transformer进行微小修改。通过消除复杂蒸馏目标，统一设计简化了训练流程，并开启了一种新型推理策略：表示引导（Representation Guidance），利用学习到的语义信息引导和优化图像生成。在条件与无条件设置下的评估均显示，该方法在图像质量与训练收敛速度上均有显著提升，为感知表征的生成建模开辟了新方向。
原文摘要 · Abstract (English)
Latent diffusion models (LDMs) dominate high-quality image generation, yet integrating representation learning with generative modeling remains a challenge. We introduce a novel generative image modeling framework that seamlessly bridges this gap by leveraging a diffusion model to jointly model low-level image latents (from a variational autoencoder) and high-level semantic features (from a pretrained self-supervised encoder like DINO). Our latent-semantic diffusion approach learns to generate coherent image-feature pairs from pure noise, significantly enhancing both generative quality and training efficiency, all while requiring only minimal modifications to standard Diffusion Transformer architectures. By eliminating the need for complex distillation objectives, our unified design simplifies training and unlocks a powerful new inference strategy: Representation Guidance, which leverages learned semantics to steer and refine image generation. Evaluated in both conditional and unconditional settings, our method delivers substantial improvements in image quality and training convergence speed, establishing a new direction for representation-aware generative modeling. Project page and code: https://representationdiffusion.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。