提出几何字节对编码,将蛋白质结构转为可解释的离散符号序列。
Protein Structure Tokenization via Geometric Byte Pair Encoding
- 基于几何配对聚类与可微逆运动学,生成分层离散结构符号
- 压缩比超10倍,训练数据需求减少10倍以上,泛化性能稳定
- 支持功能分类对齐,适合需要可解释性的蛋白质建模研究
蛋白质结构决定生物功能,构建多模态蛋白质模型需联合推理序列、结构与功能。现有蛋白质结构分词器(PST)存在分词粒度固定或依赖连续向量码本的问题,限制了可解释性、多尺度控制与跨架构迁移。本文提出GeoBPE,一种基于几何的分词方法,将连续、噪声大、多尺度的主链构象转化为满足全局约束的离散“句子”。其核心机制包括:(i) 用k-medoids聚类几何配对,生成可调分辨率的词汇表;(ii) 将每个几何配对量化为最近的原型;(iii) 通过可微逆运动学优化边界粘合角,最小化SE(3)末端帧损失以减少漂移。GeoBPE实现超过10倍的比特/残基压缩率,训练数据需求减少10倍以上,测试/训练失真比保持在1.0–1.1之间。该方法与架构无关:(a) 分层词汇提供强归纳偏置,能从大型PLM中粗粒化残基嵌入至基序和蛋白级表示,在12项任务、24个测试划分上优于主流方法;(b) 配合Transformer可实现无需条件的主链生成;(c) 分词结果与CATH功能家族对齐,支持专家可解释的案例分析,赋予结构符号功能性语义。代码已开源。
原文摘要 · Abstract (English)
Protein structure is central to biological function, and enabling multimodal protein models requires joint reasoning over sequence, structure, and function. A key barrier is the lack of principled protein structure tokenizers (PSTs): existing approaches fix token size or rely on continuous vector codebooks, limiting interpretability, multi-scale control, and transfer across architectures. We introduce GeoBPE, a geometry-grounded PST that transforms continuous, noisy, multi-scale backbone conformations into discrete ``sentences'' of geometry while enforcing global constraints. Analogous to byte-pair encoding, GeoBPE generates a hierarchical vocabulary of geometric primitives by iteratively (i) clustering Geo-Pair occurrences with k-medoids to yield a resolution-controllable vocabulary; (ii) quantizing each Geo-Pair to its closest medoid prototype; and (iii) reducing drift through differentiable inverse kinematics that optimizes boundary glue angles under an $\mathrm{SE}(3)$ end-frame loss. GeoBPE offers compression ($>$10x reduction in bits-per-residue at similar distortion rate), data efficiency ($>$10x less training data), and generalization (maintains test/train distortion ratio of $1.0-1.1$). It is architecture-agnostic: (a) its hierarchical vocabulary provides a strong inductive bias for coarsening residue-level embeddings from large PLMs into motif- and protein-level representations, consistently outperforming leading PSTs across $12$ tasks and $24$ test splits; (b) paired with a transformer, GeoBPE supports unconditional backbone generation via language modeling; and (c) tokens align with CATH functional families and support expert-interpretable case studies, offering functional meaning absent in prior PSTs. Code is available at https://github.com/shiningsunnyday/PT-BPE/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。