arXiv:2509.00614cs.LGphysics.chem-ph2025-09NeurIPS被引 2

提出分子图模型稳健微调新方法,提升小样本场景下性能

RoFt-Mol: Benchmarking Robust Fine-Tuning with Molecular Graph Foundation Models

  • 分类八种微调方法,归纳为权重、表征与部分微调三类机制
  • 在多种标签设置下验证,新方法在回归与分类任务中均更优
  • 结合插值与集成优势,兼顾效果与使用便捷性,适合小样本研究者

在基础模型时代,针对下游任务微调预训练模型变得至关重要。这推动了应对模型过拟合和标签稀疏等挑战的稳健微调方法的发展。分子图基础模型(MGFMs)面临独特困难:预训练数据集较小,下游任务数据更稀缺,两者均需增强模型泛化能力;同时需适应回归与分类等多种目标。为更好理解并改进此类条件下的微调技术,我们将八种微调方法分为三类机制:基于权重、基于表示、部分微调。在不同标注环境下,对监督与自监督预训练模型的下游回归与分类任务进行广泛评估。该评测提供宝贵洞见,并指导设计出优化的稳健微调方法——ROFT-MOL。该方法融合简单后处理权重插值与复杂权重集成微调的优势,在保持后处理便捷性的同时,提升两类任务表现。

原文摘要 · Abstract (English)

In the era of foundation models, fine-tuning pre-trained models for specific downstream tasks has become crucial. This drives the need for robust fine-tuning methods to address challenges such as model overfitting and sparse labeling. Molecular graph foundation models (MGFMs) face unique difficulties that complicate fine-tuning. These models are limited by smaller pre-training datasets and more severe data scarcity for downstream tasks, both of which require enhanced model generalization. Moreover, MGFMs must accommodate diverse objectives, including both regression and classification tasks. To better understand and improve fine-tuning techniques under these conditions, we classify eight fine-tuning methods into three mechanisms: weight-based, representation-based, and partial fine-tuning. We benchmark these methods on downstream regression and classification tasks across supervised and self-supervised pre-trained models in diverse labeling settings. This extensive evaluation provides valuable insights and informs the design of a refined robust fine-tuning method, ROFT-MOL. This approach combines the strengths of simple post-hoc weight interpolation with more complex weight ensemble fine-tuning methods, delivering improved performance across both task types while maintaining the ease of use inherent in post-hoc weight interpolation.

分子建模微调方法小样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。