提出新指标评估图像生成多样性,发现主流模型仅达训练数据77%多样性。
Image Generation Diversity Issues and How to Tame Them
- 将多样性建模为图像检索问题,用真实图检索合成图来量化
- 现有扩散模型多样性最高仅达训练数据的77%,且通用特征提取器无效
- 设计新方法DiADM,在不损失画质前提下提升生成多样性
生成模型虽能产出接近真实的数据,却常未能充分捕捉数据分布。与质量问题不同,多样性缺陷难以通过视觉察觉,需专用度量。本文指出当前生成模型多样性不足及常用度量无法有效评估的问题。我们通过将多样性视为图像检索任务,以合成样本作为查询,衡量可检索到的真实图像数量,提出可解释、无需超参数的图像检索得分(IRS)。IRS仅需少量合成样本,提供统计置信度。实验表明,常见用于评估生成模型的特征提取器在多样性评估中表现不佳。为此,我们系统搜索最优特征提取器,结果发现当前扩散模型仅收敛至真实分布的有限子集,无一超过77%的多样性。为此,我们提出多样性感知扩散模型(DiADM),通过解耦多样性和画质,利用伪无条件特征输入的多样性感知模块,显著提升无条件扩散模型的多样性,同时保持画质。我们开源了统一特征提取与度量计算工具包,以促进生成模型评估:https://github.com/MischaD/beyondfid。
原文摘要 · Abstract (English)
Generative methods now produce outputs nearly indistinguishable from real data but often fail to fully capture the data distribution. Unlike quality issues, diversity limitations in generative models are hard to detect visually, requiring specific metrics for assessment. In this paper, we draw attention to the current lack of diversity in generative models and the inability of common metrics to measure this. We achieve this by framing diversity as an image retrieval problem, where we measure how many real images can be retrieved using synthetic data as queries. This yields the Image Retrieval Score (IRS), an interpretable, hyperparameter-free metric that quantifies the diversity of a generative model's output. IRS requires only a subset of synthetic samples and provides a statistical measure of confidence. Our experiments indicate that current feature extractors commonly used in generative model assessment are inadequate for evaluating diversity effectively. Consequently, we perform an extensive search for the best feature extractors to assess diversity. Evaluation reveals that current diffusion models converge to limited subsets of the real distribution, with no current state-of-the-art models superpassing 77% of the diversity of the training data. To address this limitation, we introduce Diversity-Aware Diffusion Models (DiADM), a novel approach that improves diversity of unconditional diffusion models without loss of image quality. We do this by disentangling diversity from image quality by using a diversity aware module that uses pseudo-unconditional features as input. We provide a Python package offering unified feature extraction and metric computation to further facilitate the evaluation of generative models https://github.com/MischaD/beyondfid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。