用1小时微调ESM2，性能接近端到端训练的ESM3。
InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions
- 仅用序列预训练模型ESM2，通过结构信息微调提升预测能力。
- 微调后模型在突变效应预测上达到与ESM3相当的准确率。
- 揭示融合方法和调优策略对性能影响显著，适合资源有限的研究者。
多模态蛋白质语言模型在突变效应预测上表现优异，但从头训练需巨大算力。本文提出InstructPLM-mu微调框架，探讨：仅用序列预训练的模型，通过结构输入微调能否达到端到端训练模型的性能？实验表明，对ESM2进行结构信息微调，可实现与ESM3相当的预测性能。我们系统比较了三种特征融合设计和微调策略，结果表明融合方式与调优方案均显著影响最终准确率，说明微调过程并非简单操作。本工作为向预训练蛋白模型注入结构信息提供实用指导，并推动更优融合机制与调优协议的研究。
原文摘要 · Abstract (English)
Multimodal protein language models deliver strong performance on mutation-effect prediction, but training such models from scratch demands substantial computational resources. In this paper, we propose a fine-tuning framework called InstructPLM-mu and try to answer a question: \textit{Can multimodal fine-tuning of a pretrained, sequence-only protein language model match the performance of models trained end-to-end? } Surprisingly, our experiments show that fine-tuning ESM2 with structural inputs can reach performance comparable to ESM3. To understand how this is achieved, we systematically compare three different feature-fusion designs and fine-tuning recipes. Our results reveal that both the fusion method and the tuning strategy strongly affect final accuracy, indicating that the fine-tuning process is not trivial. We hope this work offers practical guidance for injecting structure into pretrained protein language models and motivates further research on better fusion mechanisms and fine-tuning protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。