提出新评估方法PGD,让图生成模型对比更可靠。
PolyGraph Discrepancy: a classifier-based metric for graph generation
- 用二分类器区分真实与生成图,计算分布距离
- 结果在[0,1]区间内,跨描述符可比
- 提供理论保证的综合评估指标,适合模型对比
现有图生成模型评估多依赖基于图描述符的MMD度量,虽能排序模型性能,但无绝对评价标准,且对核函数和描述符参数敏感,导致不同描述符间不可比。本文提出PolyGraph Discrepancy(PGD),通过拟合二分类器区分真实与生成图(以描述符表征),其数据对数似然近似变分下界,估计两分布间的Jensen-Shannon距离。所得度量被约束在[0,1]区间,可在不同描述符间直接比较。进一步推导出理论支持的汇总度量,为给定描述符提供最紧致的下界。大量实验表明,PGD相比MMD更具鲁棒性与洞察力。图生成模型评测框架PolyGraph已开源:https://github.com/BorgwardtLab/polygraph-benchmark。
原文摘要 · Abstract (English)
Existing methods for evaluating graph generative models primarily rely on Maximum Mean Discrepancy (MMD) metrics based on graph descriptors. While these metrics can rank generative models, they do not provide an absolute measure of performance. Their values are also highly sensitive to extrinsic parameters, namely kernel and descriptor parametrization, making them incomparable across different graph descriptors. We introduce PolyGraph Discrepancy (PGD), a new evaluation framework that addresses these limitations. It approximates the Jensen-Shannon distance of graph distributions by fitting binary classifiers to distinguish between real and generated graphs, featurized by these descriptors. The data log-likelihood of these classifiers approximates a variational lower bound on the JS distance between the two distributions. Resulting metrics are constrained to the unit interval [0,1] and are comparable across different graph descriptors. We further derive a theoretically grounded summary metric that combines these individual metrics to provide a maximally tight lower bound on the distance for the given descriptors. Thorough experiments demonstrate that PGD provides a more robust and insightful evaluation compared to MMD metrics. The PolyGraph framework for benchmarking graph generative models is made publicly available at https://github.com/BorgwardtLab/polygraph-benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。