用跨层转换器揭示蛋白质模型的计算电路，发现关键功能模块。
Protein Circuit Tracing via Cross-layer Transcoders
- 通过跨层转换器联合学习多层稀疏表示，捕捉完整计算路径。
- 在蛋白家族分类任务中保留82%-89%原始性能，压缩后仅用1%空间仍达79%准确率。
- 可识别结合、信号传递等结构功能模块，用于高效蛋白设计。
蛋白质语言模型（pLMs）已成为预测蛋白质结构与功能的强大工具，但其内部计算机制仍不清晰。现有可解释性方法独立处理各层表示，难以捕捉跨层计算过程，限制了对模型全貌的逼近能力。本文提出ProtoMech框架，利用跨层转换器联合学习多层稀疏潜在表示，以揭示pLM的完整计算电路。应用于ESM2模型时，ProtoMech在蛋白家族分类和功能预测任务中恢复了82%-89%的原始性能；进一步识别出仅使用<1%潜在空间的压缩电路，仍保持最高79%的模型准确率，并与结合、信号传导和稳定性等结构功能基序对应。沿这些电路进行调控可实现高适配度蛋白设计，在超过70%案例中优于基线方法。结果表明ProtoMech是蛋白质计算电路追踪的系统性框架。
原文摘要 · Abstract (English)
Protein language models (pLMs) have emerged as powerful predictors of protein structure and function. However, the computational circuits underlying their predictions remain poorly understood. Recent mechanistic interpretability methods decompose pLM representations into interpretable features, but they treat each layer independently and thus fail to capture cross-layer computation, limiting their ability to approximate the full model. We introduce ProtoMech, a framework for discovering computational circuits in pLMs using cross-layer transcoders that learn sparse latent representations jointly across layers to capture the model's full computational circuitry. Applied to the pLM ESM2, ProtoMech recovers 82-89% of the original performance on protein family classification and function prediction tasks. ProtoMech then identifies compressed circuits that use <1% of the latent space while retaining up to 79% of model accuracy, revealing correspondence with structural and functional motifs, including binding, signaling, and stability. Steering along these circuits enables high-fitness protein design, surpassing baseline methods in more than 70% of cases. These results establish ProtoMech as a principled framework for protein circuit tracing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。