首个系统性评测分子属性外分布预测能力的基准，揭示当前模型泛化短板。
BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
- 构建化学合理基准，评估模型在分子属性外分布预测中的表现
- 顶级模型外分布误差是内分布的3倍，普遍泛化能力不足
- 适合关注分子生成与模型鲁棒性的计算化学研究者
数据驱动的分子发现依赖人工智能/机器学习（AI/ML）与生成建模来筛选和设计新分子。新分子发现需要准确的外分布（OOD）预测能力，但现有机器学习模型在跨分布泛化上表现不佳。目前尚无系统性的分子OOD预测任务基准。我们提出BOOM——一个针对常见分子属性预测任务的化学知情外分布性能基准。我们评估了超过150种模型-任务组合，对深度学习模型在OOD性能上的表现进行系统评估。总体发现：无现有模型在所有任务中均实现强泛化能力；即使表现最优的模型，其平均外分布误差也达到内分布的3倍。当前化学基础模型未展现出强外推能力，而具有高归纳偏置的模型可在简单特定属性任务上表现良好。我们进行了广泛的消融实验，揭示数据生成、预训练、超参数优化、模型架构和分子表示对OOD性能的影响。开发具备强外分布泛化能力的模型，是化学机器学习领域的新前沿挑战。该开源基准已发布于https://github.com/FLASK-LLNL/BOOM。
原文摘要 · Abstract (English)
Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present $\mathbf{BOOM}$, $\mathbf{b}$enchmarks for $\mathbf{o}$ut-$\mathbf{o}$f-distribution $\mathbf{m}$olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。