为分子结构动态生成适配权重,让大模型精准理解每个分子的特性。
MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistant
- 基于分子结构实时生成独特低秩适配参数,动态注入冻结大模型。
- 在反应预测任务上准确率提升14.1%,量子性质预测误差降低22%。
- 适合需要精细分子结构理解的药物发现场景,避免模型遗忘通用能力。
在药物发现中,如何有效融合分子图结构与大语言模型(LLM)是关键挑战。现有方法通常通过微调LLM或添加静态适配器来处理分子结构,但存在两大问题:一是所有分子共享同一参数空间,难以捕捉实例特异性结构特征;二是微调可能导致灾难性遗忘,削弱模型通用推理能力。本文提出一种即刻生成的分子感知低秩适配框架(MoRA),为每个输入分子图动态生成唯一低秩适配权重,并将其注入冻结的LLM中,实现对每种分子结构的个性化推理,同时保留模型核心知识。大量实验表明,在化学反应预测和分子描述生成等关键任务上,MoRA的实例化动态适配优于静态适配基线,反应预测精确匹配率相对提升14.1%,量子性质预测误差减少22%。代码已开源。
原文摘要 · Abstract (English)
Effectively integrating molecular graph structures with Large Language Models (LLMs) is a key challenge in drug discovery. Most existing multi-modal alignment methods typically process these structures by fine-tuning the LLM or adding a static adapter simultaneously. However, these approaches have two main limitations: (1) it optimizes a shared parameter space across all molecular inputs, limiting the model's ability to capture instance-specific structural features; and (2) fine-tuning the LLM for molecular tasks can lead to catastrophic forgetting, undermining its general reasoning capabilities. In this paper, instead of static task-oriented adaptation, we propose an instance-specific parameter space alignment approach for each molecule on-the-fly. To this end, we introduce Molecule-aware Low-Rank Adaptation (MoRA) that produces a unique set of low-rank adaptation weights for each input molecular graph. These weights are then dynamically injected into a frozen LLM, allowing the model to adapt its reasoning to the structure of each molecular input, while preserving the LLM's core knowledge. Extensive experiments demonstrate that on key molecular tasks, such as chemical reaction prediction and molecular captioning, MoRA's instance-specific dynamic adaptation outperforms statically adapted baselines, including a 14.1% relative improvement in reaction prediction exact match and a 22% reduction in error for quantum property prediction. The code is available at https://github.com/jk-sounds/MoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。