用低秩适配降低大模型训练成本，让小模型也能精准预测蛋白属性。
SeqProFT: Sequence-only Protein Property Prediction with LoRA Finetuning
- 仅通过序列数据+LoRA微调，大幅减少计算开销。
- 小模型经LoRA优化后性能超过未优化的大模型。
- 适合算力有限但需高精度蛋白属性预测的研究者。
蛋白质语言模型（PLMs）在学习蛋白质序列与功能关系方面展现出强大能力。然而，微调这些大型模型需要大量计算资源，且任务特定表现常不理想。本研究探讨了通过低秩适配（LoRA）实现参数高效微调，以提升蛋白质属性预测性能并显著降低计算需求。我们在不同规模的ESM-2和ESM-C模型上应用LoRA，评估了10项多样化的蛋白质属性预测任务。结果表明，经LoRA适配的小模型性能可达到甚至超越未经适配的大模型。此外，通过多头注意力机制融合接触图信息，提升了模型对结构特征的理解。系统分析显示，LoRA微调能实现更快收敛、更优性能及更高资源利用效率，为资源受限环境下的蛋白质研究提供实用指导。代码已公开于https://github.com/jiankliu/SeqProFT。
原文摘要 · Abstract (English)
Protein language models (PLMs) have demonstrated remarkable capabilities in learning relationships between protein sequences and functions. However, finetuning these large models requires substantial computational resources, often with suboptimal task-specific results. This study investigates how parameter-efficient finetuning via LoRA can enhance protein property prediction while significantly reducing computational demands. By applying LoRA to ESM-2 and ESM-C models of varying sizes and evaluating 10 diverse protein property prediction tasks, we demonstrate that smaller models with LoRA adaptation can match or exceed the performance of larger models without adaptation. Additionally, we integrate contact map information through a multi-head attention mechanism, improving model comprehension of structural features. Our systematic analysis reveals that LoRA finetuning enables faster convergence, better performance, and more efficient resource utilization, providing practical guidance for protein research applications in resource-constrained environments. The code is available at https://github.com/jiankliu/SeqProFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。