提出新评估方法,专门检验模型在罕见分子结构上的生成能力。
Vertical Validation: Evaluating Implicit Generative Models for Graphs on Thin Support Regions
- 通过特征依赖的训练测试划分,在稀疏区域构建细支持集
- 重加权生成样本以实现与测试数据的公平比较
- 适合关注新分子发现、防过拟合的药物/材料设计研究者
隐式图生成模型在药物和材料设计中被寄予厚望,因其可生成尚未发现的分子,这些分子天然位于已知分子分布的未探索或稀疏区域。然而,现有评估方法主要针对密集支持区(如图属性的均值和方差)进行统计验证,导致生成目标与评估方法不匹配。为此,本文提出垂直验证(Vertical Validation, VV),在训练-测试划分过程中系统构建细支持区域,并对生成样本进行重加权,使其可与保留的测试数据进行有效对比。该方法可视为标准训练-测试流程的推广,但分割依赖于样本特征。实验表明,该方法可用于以稀疏区域性能为目标的模型选择;同时,还能更有效地检测过拟合现象,如记忆行为。
原文摘要 · Abstract (English)
There has been a growing excitement that implicit graph generative models could be used to design or discover new molecules for medicine or material design. Because these molecules have not been discovered, they naturally lie in unexplored or scarcely supported regions of the distribution of known molecules. However, prior evaluation methods for implicit graph generative models have focused on validating statistics computed from the thick support (e.g., mean and variance of a graph property). Therefore, there is a mismatch between the goal of generating novel graphs and the evaluation methods. To address this evaluation gap, we design a novel evaluation method called Vertical Validation (VV) that systematically creates thin support regions during the train-test splitting procedure and then reweights generated samples so that they can be compared to the held-out test data. This procedure can be seen as a generalization of the standard train-test procedure except that the splits are dependent on sample features. We demonstrate that our method can be used to perform model selection if performance on thin support regions is the desired goal. As a side benefit, we also show that our approach can better detect overfitting as exemplified by memorization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。