用视觉质量评估修复生成模型的低质图像缺陷
Images Speak Louder Than Scores: Failure Mode Escape for Enhancing Generative Quality
- 通过质量评估模型识别低质生成样本并记录采样轨迹
- 利用失败模式作为负向引导，提升生成图像的感知质量
- 无需训练且适用于文本到图像生成，适合质量敏感场景
扩散模型在类到图像生成上取得显著进展，但即使FID分数优异，顶尖模型仍常生成失真或低质量图像，尤其在特定类别中。这是因为FID仅评估全局分布对齐，忽略单个样本的感知质量。我们进一步分析了常用的质量增强技术CFG，发现其虽能提升指标并抑制异常值，但可能因与训练目标和用户期望不一致，引入分布偏移和视觉伪影。本文提出无训练、推理高效的FaME方法：使用图像质量评估模型识别低质生成结果，存储其采样轨迹，并以此作为负向引导，使后续采样避开低质区域。ImageNet实验表明，FaME在不损害FID的前提下持续提升视觉质量，且具有扩展至文生图任务的潜力。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable progress in class-to-image generation. However, we observe that despite impressive FID scores, state-of-the-art models often generate distorted or low-quality images, especially in certain classes. This gap arises because FID evaluates global distribution alignment, while ignoring the perceptual quality of individual samples. We further examine the role of CFG, a common technique used to enhance generation quality. While effective in improving metrics and suppressing outliers, CFG can introduce distribution shift and visual artifacts due to its misalignment with both training objectives and user expectations. In this work, we propose FaME, a training-free and inference-efficient method for improving perceptual quality. FaME uses an image quality assessment model to identify low-quality generations and stores their sampling trajectories. These failure modes are then used as negative guidance to steer future sampling away from poor-quality regions. Experiments on ImageNet demonstrate that FaME brings consistent improvements in visual quality without compromising FID. FaME also shows the potential to be extended to improve text-to-image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。