提出新方法估算生成模型的完整精确率与召回率曲线。
A New Perspective on Precision and Recall for Generative Models
- 从二分类视角构建全新PR曲线估计框架。
- 给出最小最大上界，理论保障估计风险可控。
- 可统一多个经典度量，适合评估生成质量与多样性。
随着生成模型在图像和文本领域的成功，其评估问题日益受到关注。尽管现有方法多依赖标量指标，但精确率与召回率（PR）的引入开辟了新的研究方向。PR曲线能提供更丰富的分析视角，但其估计面临诸多挑战。本文提出一种基于二分类视角的新框架，用于估计完整的PR曲线，并对其估计量进行深入的统计分析。作为副产品，我们获得了PR估计风险的最小最大上界。此外，证明该框架可扩展文献中多个标志性PR度量，这些度量因设计限制仅适用于曲线极值点。最后，我们在多种实验设置下研究了曲线的实际行为差异。
原文摘要 · Abstract (English)
With the recent success of generative models in image and text, the question of their evaluation has recently gained a lot of attention. While most methods from the state of the art rely on scalar metrics, the introduction of Precision and Recall (PR) for generative model has opened up a new avenue of research. The associated PR curve allows for a richer analysis, but their estimation poses several challenges. In this paper, we present a new framework for estimating entire PR curves based on a binary classification standpoint. We conduct a thorough statistical analysis of the proposed estimates. As a byproduct, we obtain a minimax upper bound on the PR estimation risk. We also show that our framework extends several landmark PR metrics of the literature which by design are restrained to the extreme values of the curve. Finally, we study the different behaviors of the curves obtained experimentally in various settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。