arXiv:2602.14682cs.LGcs.AI2026-02被引 1

发现生成模型普遍存在多样性低估问题,揭示其统计成因并提出校正方向。

Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error

  • 用无参考熵值评分直接对比真实数据与生成样本的多样性。
  • 实测显示生成样本多样性显著低于真实测试数据,存在系统性低估。
  • 提出基于熵值的正则化策略,适合关注生成多样性研究者。

深度生成模型在生成高质量样本方面取得巨大成功,已成为机器学习应用的核心工具。除了样本质量,一个较少系统研究的问题是:训练后的生成模型是否准确捕捉了底层数据分布的多样性?本文通过近期提出的无参考熵基多样性评分指标 Vendi 和 RKE,直接比较了先进模型生成样本与目标数据分布中测试样本的多样性。在多个基准数据集上,测试数据的 Vendi 与 RKE 分数均显著高于生成样本,表明现代生成模型存在系统性的多样性低估偏差。为理解该偏差根源,我们分析了熵基多样性评分的有限样本行为,发现其期望值随样本量增加而上升,这意味着从有限训练集估计的多样性会天然低估真实分布的多样性。因此,优化生成器以最小化与经验数据分布的差异,会诱导多样性损失。最后,我们讨论基于 Vendi 与 RKE 的多样性感知正则化和引导策略,作为缓解该偏差的合理方向,并提供了实证证据支持其改进潜力。

原文摘要 · Abstract (English)

Deep generative models have achieved great success in producing high-quality samples, making them a central tool across machine learning applications. Beyond sample quality, an important yet less systematically studied question is whether trained generative models faithfully capture the diversity of the underlying data distribution. In this work, we address this question by directly comparing the diversity of samples generated by state-of-the-art models with that of test samples drawn from the target data distribution, using recently proposed reference-free entropy-based diversity scores, Vendi and RKE. Across multiple benchmark datasets, we find that test data consistently attains substantially higher Vendi and RKE diversity scores than the generated samples, suggesting a systematic downward diversity bias in modern generative models. To understand the origin of this bias, we analyze the finite-sample behavior of entropy-based diversity scores and show that their expected values increase with sample size, implying that diversity estimated from finite training sets could inherently underestimate the diversity of the true distribution. As a result, optimizing the generators to minimize divergence to empirical data distributions would induce a loss of diversity. Finally, we discuss potential diversity-aware regularization and guidance strategies based on Vendi and RKE as principled directions for mitigating this bias, and provide empirical evidence suggesting their potential to improve the results.

生成模型多样性偏差熵基评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。