arXiv:2605.15309cs.CV2026-05

递归优化潜空间,同时提升图像生成的质量与多样性。

One Pass Is Not Enough: Recursive Latent Refinement for Generative Models

论文配图:One Pass Is Not Enough: Recursive Latent Refinement for Generative Models
图 1 · 摘自论文原文
  • 用迭代精炼替代单次潜变量映射,增强分布覆盖能力
  • 在多个数据集上实现最高精度与召回率,FID保持领先
  • 适合追求生成多样性的研究者和工业应用

尽管图像生成取得显著进展,但主流指标FID已接近饱和,无法有效区分样本保真度与模式覆盖。模型可能因少量高保真重复图像而获得低FID,却存在模式坍缩问题。本文主张引入精确率与召回率作为补充指标,并聚焦提升多样性与覆盖范围。提出RTM方法,将风格化生成器中的单次潜变量映射替换为递归精炼过程,结合隐式最大似然估计(IMLE)以显式优化模式覆盖。在CIFAR-10、CelebA-HQ(256×256)、九个少样本基准上,RTM实现了当前最佳的精确率与召回率,同时保持竞争力的FID;在CIFAR-10与AFHQ-v1(512×512)上亦改进了StyleGAN2与StyleGAN2-ADA。相比流匹配方法虽有良好FID却牺牲覆盖的缺点,递归精炼可同步提升质量与多样性。

原文摘要 · Abstract (English)

Despite remarkable progress, image generation is far from solved. The dominant metric, FID, conflates sample fidelity with mode coverage and is close to being saturated. Yet a model can still exhibit mode collapse while achieving a low FID, since a handful of sharp, near-duplicate images can outscore a model that faithfully covers the full data distribution. We argue that precision and recall are essential complements to FID, and that because FID is already saturated, the more meaningful goal is to improve diversity and coverage. Achieving high recall requires a model that explicitly prioritizes mode coverage, unlike most generative models, which optimize sample fidelity. We introduce RTM, which replaces the single-pass latent mapping in style-based generators with an iterative refinement process, and show that this consistently improves both quality and diversity. Integrated with Implicit Maximum Likelihood Estimation (IMLE), which optimizes mode coverage by design, RTM achieves the highest precision and recall among current state-of-the-art approaches while maintaining competitive FID, with improvements across CIFAR-10, CelebA-HQ at 256x256, and nine few-shot benchmarks. RTM also improves StyleGAN2 and StyleGAN2-ADA on CIFAR-10 and AFHQ-v1 at 512x512, demonstrating that the benefit is not specific to IMLE. Unlike flow-matching baselines that achieve competitive FID at the expense of coverage, recursive refinement improves both quality and diversity simultaneously.

图像生成扩散模型多样性潜空间优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。