arXiv:2605.09981q-bio.BMcs.AI2026-05

Yeti用小模型实现高精度蛋白结构编码,支持序列与结构联合生成。

Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation

论文配图:Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation
图 1 · 摘自论文原文
  • 基于无查表量化,端到端训练生成结构离散表示。
  • 参数量仅ESM3的十分之一,重建准确率第二,代码本利用率最高。
  • 可训练小型多模态模型从零生成合理蛋白结构和序列。

联合建模蛋白序列、结构和功能注释的多模态模型在整合多源数据和设计新功能蛋白方面潜力巨大。为适配Transformer架构,此类模型需将连续原子坐标转换为适合大规模训练的离散表示。模型性能受限于底层结构编码的保真度与表达力。现有编码器更侧重重建而非生成能力。为此,我们提出Yeti,一种基于无查表量化、端到端训练的轻量级蛋白结构编码器,采用流匹配目标优化多模态学习。相较于现有方法,Yeti在多个数据集上实现最优代码本利用率与令牌多样性,重建精度位居第二(参数量仅为ESM3的1/10)。为验证其生成能力,我们从零开始训练了一个紧凑的多模态模型,联合使用其结构令牌与氨基酸序列。该模型在无条件联合生成中可生成合理结构,表现接近10倍大的模型。结果表明,Yeti是一种适用于联合生成高可信度序列与结构的紧凑且表达力强的蛋白结构编码器。

原文摘要 · Abstract (English)

Multimodal models that jointly reason over protein sequences, structures, and function annotations within a unified representation hold immense potential for integrating multimodal data and generating new proteins with designed functional properties. To utilize transformer architectures, such models require a tokenizer that converts protein structure from continuous atomic coordinates into discrete representations suitable for scalable multimodal training. The quality of such models are fundamentally upper bounded by the fidelity and expressiveness of the underlying tokenized structure. However, existing tokenizers prioritize reconstruction over generative abilities. To address these gaps, we introduce Yeti, a simple and compact protein structure tokenizer based on lookup free quantization and trained end to end with a flow matching objective for multimodal learning. Compared to existing models, Yeti generally achieves the best codebook utilization and token diversity, and second best reconstruction accuracy (with 10x fewer parameters than ESM3) on diverse datasets. To validate Yeti's generative capability, we trained a compact multimodal model jointly over its structure tokens and amino acid sequence entirely from scratch, with no pretrained initialization. The resulting multimodal model generates plausible structures under unconditional cogeneration of protein sequence and structures, achieving comparable results to 10x larger models. Together, these results demonstrate that Yeti is a compact and expressive protein structure tokenizer suitable for training multimodal models that cogenerates highly plausible sequences and structures.

蛋白生成结构编码多模态学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。