用多模态数据增强指令微调,提升大模型生成多属性分子的能力。
Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Models
- 用文本、SMILES和化学性质对齐训练,合成高质量指令数据。
- 在分子描述生成等任务上超越MolT5等基线模型,多属性生成效果显著。
- 适合需要精准控制分子特性的药物研发与生成式化学研究者。
大语言模型(LLMs)在自然语言处理中广泛应用,但因生化性质标注数据稀缺且人工标注困难,其在分子生成任务中的表现受限,尤其在多属性约束场景下。本文提出两阶段框架PEIT(Property Enhanced Instruction Tuning),以提升LLMs在分子相关任务中的性能。第一阶段,利用文本描述、SMILES和生化性质作为多模态输入,通过对齐多模态表示来合成指令数据,预训练得到PEIT-GEN模型;第二阶段,使用合成数据微调开源大模型,得到的PEIT-LLM可处理分子描述生成、基于文本的分子生成、分子性质预测及新提出的多约束分子生成任务。实验表明,预训练的PEIT-GEN在分子描述生成任务上优于MolT5、BioT5、MolCA和Text+Chem-T5,验证了文本、结构与生化性质之间的模态对齐有效。此外,PEIT-LLM在多任务分子生成中表现优异,证明了该框架在分子任务上的有效性。代码与附录见https://github.com/chenlong164/PEIT。
原文摘要 · Abstract (English)
Large language models (LLMs) are widely applied in various natural language processing tasks such as question answering and machine translation. However, due to the lack of labeled data and the difficulty of manual annotation for biochemical properties, the performance for molecule generation tasks is still limited, especially for tasks involving multi-properties constraints. In this work, we present a two-step framework PEIT (\textbf{P}roperty \textbf{E}nhanced \textbf{I}nstruction \textbf{T}uning) to improve LLMs for molecular-related tasks. In the first step, we use textual descriptions, SMILES, and biochemical properties as multimodal inputs to pre-train a model called PEIT-GEN, by aligning multi-modal representations to synthesize instruction data. In the second step, we fine-tune existing open-source LLMs with the synthesized data, the resulting PEIT-LLM can handle molecule captioning, text-based molecule generation, molecular property prediction, and our newly proposed multi-constraint molecule generation tasks. Experimental results show that our pre-trained PEIT-GEN outperforms MolT5, BioT5, MolCA and Text+Chem-T5 in molecule captioning, demonstrating modalities align well between textual descriptions, structures, and biochemical properties. Furthermore, PEIT-LLM shows promising improvements in multi-task molecule generation, demonstrating the effectiveness of the PEIT framework for molecular tasks. The code and appendix are available at https://github.com/chenlong164/PEIT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。