25个分子嵌入模型对比发现，多数不如传统指纹方法。
Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning
- 在25个数据集上公平比较25种预训练分子模型
- 仅CLAMP模型显著优于基础的ECFP指纹方法
- 揭示现有研究评估方式可能过于乐观
预训练神经网络在化学与小分子药物设计中备受关注。此类模型生成的嵌入广泛用于分子性质预测、虚拟筛选和小样本学习。本研究迄今最全面地比较了25种分子嵌入模型，在25个数据集上评估其表现。在公平比较框架下，我们考察了不同模态、架构和预训练策略的模型。通过专用的分层贝叶斯统计检验模型，发现几乎所有神经模型相较于基线的ECFP分子指纹均无显著提升。唯一显著更优的是基于分子指纹的CLAMP模型。该结果引发对现有研究评估严谨性的担忧。我们讨论潜在原因，提出改进方案，并给出实用建议。
原文摘要 · Abstract (English)
Pretrained neural networks have attracted significant interest in chemistry and small molecule drug design. Embeddings from these models are widely used for molecular property prediction, virtual screening, and small data learning in molecular chemistry. This study presents the most extensive comparison of such models to date, evaluating 25 models across 25 datasets. Under a fair comparison framework, we assess models spanning various modalities, architectures, and pretraining strategies. Using a dedicated hierarchical Bayesian statistical testing model, we arrive at a surprising result: nearly all neural models show negligible or no improvement over the baseline ECFP molecular fingerprint. Only the CLAMP model, which is also based on molecular fingerprints, performs statistically significantly better than the alternatives. These findings raise concerns about the evaluation rigor in existing studies. We discuss potential causes, propose solutions, and offer practical recommendations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。