arXiv:2410.21719stat.MLcs.AI2024-10被引 13

提出截断版Vendi分数,解决小样本下多样性评估不收敛问题。

Do Vendi Scores Converge with Finite Samples? Truncated Vendi Score for Finite-Sample Convergence Guarantees

  • 通过截断核矩阵特征谱,设计可证明收敛的t-截断Vendi分数
  • 在20,000样本以下时,标准Vendi分数常不收敛,而新方法保证收敛
  • 适合图像与文本生成模型的多样性评估,尤其适用于小样本场景

无参考数据下评估生成模型多样性存在方法论挑战。参考无关的Vendi和RKE分数通过基于矩阵的熵度量来量化生成数据的多样性。其中,Vendi分数通常通过n个生成样本构建的n×n核矩阵进行特征分解计算,但当n较大时特征分解成本过高,常限制样本数少于20,000。本文研究了有限样本下Vendi和RKE分数的统计收敛性,数值表明标准Vendi分数在样本量低于20,000时通常无法收敛至无限采样下的极限值。为此,我们提出t-截断Vendi分数,通过对核矩阵特征谱截断,证明其在n=O(t)样本下可收敛至总体极限。进一步表明,现有Nyström和FKEA近似方法收敛于截断后Vendi分数的渐近极限。相比之下,我们证明RKE分数对所有核函数均具有普适收敛性。通过数值实验展示了Nyström和FKEA计算的Vendi分数围绕截断版本的集中性,并分析了截断后的Vendi与RKE分数与图像及文本数据多样性的相关性。代码已公开于https://github.com/aziksh-ospanov/truncated-vendi。

原文摘要 · Abstract (English)

Evaluating the diversity of generative models without reference data poses methodological challenges. The reference-free Vendi and RKE scores address this by quantifying the diversity of generated data using matrix-based entropy measures. Among these two, the Vendi score is typically computed via the eigendecomposition of an $n \times n$ kernel matrix constructed from n generated samples. However, the prohibitive computational cost of eigendecomposition for large $n$ often limits the number of samples used to fewer than 20,000. In this paper, we investigate the statistical convergence of the Vendi and RKE scores under restricted sample sizes. We numerically demonstrate that, in general, the Vendi score computed with standard sample sizes below 20,000 may not converge to its asymptotic value under infinite sampling. To address this, we introduce the $t$-truncated Vendi score by truncating the eigenspectrum of the kernel matrix, which is provably guaranteed to converge to its population limit with $n=\mathcal{O}(t)$ samples. We further show that existing Nyström and FKEA approximation methods converge to the asymptotic limit of the truncated Vendi score. In contrast to the Vendi score, we prove that the RKE score enjoys universal convergence guarantees across all kernel functions. We conduct several numerical experiments to illustrate the concentration of Nyström and FKEA computed Vendi scores around the truncated Vendi score, and we analyze how the truncated Vendi and RKE scores correlate with the diversity of image and text data. The code is available at https://github.com/aziksh-ospanov/truncated-vendi.

多样性评估生成模型收敛性核方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。