让蛋白模型更懂功能与结构相似性,提升下游任务表现
ProtSent: Protein Sentence Transformers

- 用对比学习微调蛋白语言模型,优化嵌入空间结构
- 在23个任务中15项提升,远程同源检测准确率翻倍
- 无需特定任务标注,适合蛋白质功能预测研究者
蛋白语言模型(pLMs)生成残基级表示,捕捉进化与结构信息,但其均值池化序列嵌入未显式训练以反映蛋白间功能、进化或结构相似性。我们提出Protein Sentence Transformers(ProtSent),一种用于将pLMs转为通用嵌入模型的对比微调框架。ProtSent在五个蛋白对数据集上使用MultipleNegativesRankingLoss进行训练:Pfam家族、结构推导的难负样本、AlphaFold DB结构对、StringDB蛋白-蛋白相互作用以及深突变扫描数据。我们在23个下游任务上使用冻结嵌入和k近邻探测器评估嵌入邻域质量。对于ESM-2 150M,ProtSent在23项任务中有15项表现提升,其中远程同源检测提升+105%,变异效应预测提升+17%,SCOPe-40结构检索的Recall@1提升+19.9%。35M版本在23项任务中有16项提升,远程同源检测提升+40.5%,SCOPe-40 Recall@1提升+15.5%。对比微调重构了嵌入空间,更好捕捉蛋白功能与结构,且无需任何任务特定监督。我们发布了模型、公开数据及训练代码。
原文摘要 · Abstract (English)
Protein language models (pLMs) produce per-residue representations that capture evolutionary and structural information, yet their mean-pooled sequence embeddings are not explicitly trained to reflect functional, evolutionary or structural similarity between proteins. We present Protein Sentence Transformers (ProtSent), a contrastive fine-tuning framework for adapting PLMs into general-purpose embedding models. ProtSent trains with MultipleNegativesRankingLoss across five protein-pair datasets: Pfam families, structurally derived hard negatives, AlphaFold DB structural pairs, and StringDB protein--protein interactions, and Deep Mutational Scanning data. We evaluate on 23~downstream tasks using frozen embeddings with a k-nearest-neighbor probe to measure embedding neighborhood quality. On ESM-2 150M, ProtSent improves 15 of 23 tasks, with gains of +105% on remote homology detection, +17% on variant effect prediction, and +19.9% Recall@1 on SCOPe-40 structural retrieval. The 35M variant improves 16 of 23 tasks with +40.5% on remote homology and +15.5% Recall@1 on SCOPe-40. Contrastive fine-tuning restructures the embedding space to better capture protein function and structure, without any task-specific supervision. We release the models, public data, and training recipe and code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。