统一蛋白预测任务,用生成式模型加速科研发现
Prot2Token: A Unified Framework for Protein Modeling via Next-Token Prediction
- 将各类蛋白预测转为统一的下一步词预测格式
- 3D结构预测快1000倍,其他任务性能媲美专用模型
- 适合需要高效多任务蛋白建模的研究者
蛋白预测任务种类繁多,传统上需定制化模型,限制了通用且高效的蛋白语言模型(PLMs)发展。本文提出Prot2Token,通过将序列属性、残基特征到蛋白间相互作用等多样任务统一为标准化的下一步词预测形式,克服这一挑战。其核心采用自回归解码器,基于预训练蛋白编码器嵌入,并由可学习的任务标记引导,实现多任务学习,使通用解码器在五类不同任务中实现泛化。在多个基准测试中验证了其强大预测能力:在3D结构预测上,速度比AlphaFold2(含MSA)快达1000倍(同硬件条件下),其他任务表现匹配或超越专用方法。此外,引入辅助自监督解码器预训练策略,进一步提升空间敏感任务性能。Prot2Token推动生物预测向生成式接口标准化迈进,有望加速生物发现与新药研发。代码已开源:https://github.com/mahdip72/prot2token。
原文摘要 · Abstract (English)
The diverse nature of protein prediction tasks has traditionally necessitated specialized models, hindering the development of broadly applicable and computationally efficient Protein Language Models (PLMs). In this work, we introduce Prot2Token, a unified framework that overcomes these challenges by converting a wide spectrum of protein-related predictions-from sequence-level properties and residue-specific attributes to complex inter-protein interactions-into a standardized next-token prediction format. At its core, Prot2Token employs an autoregressive decoder, conditioned on embeddings from pre-trained protein encoders and guided by learnable task tokens, to perform diverse predictions. This architecture uniquely facilitates multi-task learning, enabling general-purpose decoders to generalize across five distinct categories. We present extensive experimental validation across a variety of benchmarks, demonstrating Prot2Token's predictive power in different types of protein-prediction tasks. In 3D structure prediction, Prot2Token delivers substantial speedups (up to 1000x faster than AlphaFold2 with MSA on the same hardware) while, across other numerous tasks, matching or surpassing specialized methods. Beyond that, we introduce an auxiliary self-supervised decoder pre-training approach to improve spatially sensitive task performance. Prot2Token thus offers a step towards standardizing biological prediction into a generative interface, promising to accelerate biological discovery and the development of novel therapeutics. The code is available at https://github.com/mahdip72/prot2token .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。