用扩散模型在不破坏嵌入结构的前提下，精准适配预训练蛋白序列表示。
Freeze, Diffuse, Decode: Geometry-Aware Adaptation of Pretrained Transformer Embeddings for Antimicrobial Peptide Design
- 冻结预训练嵌入，沿其内在流形传播监督信号
- 在抗菌肽设计任务中实现低维可解释的预测表示
- 适合小样本场景下的蛋白质序列生成与属性预测
预训练变换器提供丰富的通用嵌入，常被迁移至下游任务。然而，现有迁移策略如微调和探针法，要么扭曲预训练嵌入的几何结构，要么表达能力不足，难以捕捉任务相关信号，尤其在监督数据稀缺时问题更显著。本文提出一种基于扩散的新型框架FDD（Freeze, Diffuse, Decode），在保持嵌入底层几何结构的同时，将预训练嵌入适配至下游任务。FDD沿冻结嵌入的内在流形传播监督信号，实现嵌入空间的几何感知适应。应用于抗菌肽设计任务时，FDD生成低维、可预测且可解释的表示，支持属性预测、检索及潜在空间插值。
原文摘要 · Abstract (English)
Pretrained transformers provide rich, general-purpose embeddings, which are transferred to downstream tasks. However, current transfer strategies: fine-tuning and probing, either distort the pretrained geometric structure of the embeddings or lack sufficient expressivity to capture task-relevant signals. These issues become even more pronounced when supervised data are scarce. Here, we introduce Freeze, Diffuse, Decode (FDD), a novel diffusion-based framework that adapts pre-trained embeddings to downstream tasks while preserving their underlying geometric structure. FDD propagates supervised signal along the intrinsic manifold of frozen embeddings, enabling a geometry-aware adaptation of the embedding space. Applied to antimicrobial peptide design, FDD yields low-dimensional, predictive, and interpretable representations that support property prediction, retrieval, and latent-space interpolation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。