用图神经网络增强分子上下文学习,提升小模型性能
Graph-based Molecular In-context Learning Grounded on Morgan Fingerprints
- 用图结构对齐分子与文本描述,结合摩根指纹做局部特征匹配
- 在多个数据集上比传统方法最高提升45%准确率
- 适合资源有限场景下的分子性质预测任务
上下文学习(ICL)通过在提示中嵌入精心选择的示例,有效引导大语言模型完成分子性质预测和分子描述生成等任务,避免了繁重的微调开销。然而,现有分子ICL的提示检索方法主要依赖摩根指纹等局部特征相似性,难以捕捉分子全局结构和原子连接关系,导致推理时无法充分表征分子复杂性。此外,小型至中型语言模型因部署简单,在分子ICL领域仍被忽视。为此,我们提出自监督学习方法GAMIC(图对齐分子上下文学习),利用图神经网络(GNN)表示分子全局结构,并与文本描述对齐,同时保留摩根指纹的局部特征相似性。此外,在检索阶段引入最大边际相关性(MMR)多样性启发式策略,优化提示样本选择。实验表明,GAMIC在多个基准数据集上均显著优于基于摩根指纹的简单ICL方法,性能提升最高达45%。
原文摘要 · Abstract (English)
In-context learning (ICL) effectively conditions large language models (LLMs) for molecular tasks, such as property prediction and molecule captioning, by embedding carefully selected demonstration examples into the input prompt. This approach avoids the computational overhead of extensive pertaining and fine-tuning. However, current prompt retrieval methods for molecular tasks have relied on molecule feature similarity, such as Morgan fingerprints, which do not adequately capture the global molecular and atom-binding relationships. As a result, these methods fail to represent the full complexity of molecular structures during inference. Moreover, small-to-medium-sized LLMs, which offer simpler deployment requirements in specialized systems, have remained largely unexplored in the molecular ICL literature. To address these gaps, we propose a self-supervised learning technique, GAMIC (Graph-Aligned Molecular In-Context learning, which aligns global molecular structures, represented by graph neural networks (GNNs), with textual captions (descriptions) while leveraging local feature similarity through Morgan fingerprints. In addition, we introduce a Maximum Marginal Relevance (MMR) based diversity heuristic during retrieval to optimize input prompt demonstration samples. Our experimental findings using diverse benchmark datasets show GAMIC outperforms simple Morgan-based ICL retrieval methods across all tasks by up to 45%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。