arXiv:2412.01794cs.CVcs.AI2024-12ICCV被引 1

让生成图像更高质量,通过迁移评分模型知识实现。

IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models

  • 用IQA模型的评分结果指导扩散模型生成,学习图像与质量分数的关系。
  • 高目标质量下,多个客观指标提升最高达10%,用户偏好测试也验证效果。
  • 可灵活调节质量等级,还能反向生成退化图像,适合需要可控质量的场景。

基于扩散的生成模型虽已大幅提升图像保真度,但持续生成高质量图像仍具挑战,主要因缺乏对感知质量的有效控制机制。本文提出将图像质量评估(IQA)模型的知识引入扩散生成器,实现质量感知的图像生成。我们发现扩散模型能从IQA模型的输出及内部激活中学习复杂质量关系。首先尝试基于梯度的引导优化质量,但泛化能力有限。为此,提出IQA-Adapter框架,通过学习图像与质量分数间的隐式关联,实现对目标质量水平的条件生成。当设定高目标质量时,模型可将生成分布推向高质量子域;反之,也可作为退化模型生成逐步失真的图像。在高质条件下,IQA-Adapter在多个客观指标上提升最高达10%,用户偏好研究进一步证实其有效性,同时保持生成多样性和内容一致性。此外,扩展至参考式条件生成场景,利用IQA模型丰富的激活空间,实现跨图像的、内容无关的高质量特征迁移。

原文摘要 · Abstract (English)

Diffusion-based models have recently revolutionized image generation, achieving unprecedented levels of fidelity. However, consistent generation of high-quality images remains challenging partly due to the lack of conditioning mechanisms for perceptual quality. In this work, we propose methods to integrate image quality assessment (IQA) models into diffusion-based generators, enabling quality-aware image generation. We show that diffusion models can learn complex qualitative relationships from both IQA models' outputs and internal activations. First, we experiment with gradient-based guidance to optimize image quality directly and show this method has limited generalizability. To address this, we introduce IQA-Adapter, a novel framework that conditions generation on target quality levels by learning the implicit relationship between images and quality scores. When conditioned on high target quality, IQA-Adapter can shift the distribution of generated images towards a higher-quality subdomain, and, inversely, it can be used as a degradation model, generating progressively more distorted images when provided with a lower-quality signal. Under high-quality condition, IQA-Adapter achieves up to a 10% improvement across multiple objective metrics, as confirmed by a user preference study, while preserving generative diversity and content. Furthermore, we extend IQA-Adapter to a reference-based conditioning scenario, utilizing the rich activation space of IQA models to transfer highly specific, content-agnostic qualitative features between images.

图像生成质量评估扩散模型可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。