arXiv:2608.18597cs.LG2026-08

蛋白语言模型生成时易出现结构崩溃,新方法可低成本检测并过滤无效序列。

Off-Manifold Collapse in Guided Protein Language Models

论文配图:Off-Manifold Collapse in Guided Protein Language Models
图 1 · 摘自论文原文
  • 通过检测模型激活偏离自然分布,识别生成序列的结构失效
  • 在保持属性得分的同时,显著提升序列折叠可能性,代价极低
  • 无需修改生成器,适配多种引导方法,适合蛋白质设计实践

蛋白语言模型广泛用于蛋白质序列设计,现有方法在推理时进行控制以替代微调。但面临困境:强度太弱则无法改变性质,强度太强则生成序列逐渐难以折叠。本文揭示失败的关键特征——模型自身表征发生离流形坍缩:引导后的激活趋向于与随机氨基酸输入无异,序列复杂度下降,但属性评分器仍能给出高分。因此评分器可能未察觉崩溃,甚至因溶解性指标而奖励此类无效序列。结构与组成信息却可暴露问题。由于失败在生成后即显现,我们采用输出端检测而非修改生成器。提出一种基于自然蛋白激活统计的廉价密度先验,仅保留符合该先验的候选序列,称为马氏距离过滤。在相同引导强度下,该方法显著提升序列的结构合理性,同时维持或改善属性得分,且无需训练、不改动生成器,并可跨不同引导方式迁移。相关激活统计已公开于 https://huggingface.co/Shuibai12138/off-manifold-collapse-plm。

原文摘要 · Abstract (English)

Protein language models are widely used priors for protein sequence design, and a growing body of work controls them at inference time as an alternative to fine-tuning. Such guidance faces a dilemma: mild enough to preserve natural activation statistics, it barely moves the property; strong enough to move it, the generations become progressively harder to fold. We show the failure has a specific and cheaply detectable signature, an off-manifold collapse of the model's own representations. Guided activations fall toward a region statistically indistinguishable from random amino-acid input, and the sequences degenerate to low complexity, yet the property oracle being optimized can still score these generations as a success. The optimized oracle can therefore fail to witness the collapse and, for solubility, can actively reward it, whereas structure and composition expose the failure. Because the failure is already visible in a finished candidate, we detect it at the output rather than modify the generator. We introduce a cheap density prior over natural protein activations and keep only the candidates that remain typical under it, a training-free post-hoc step we call Mahalanobis filtering. At matched guidance settings it improves both the property score and the structural plausibility of the sequences it keeps at negligible cost, without touching the generator, and transfers across different guidance methods. We release the activation statistic at https://huggingface.co/Shuibai12138/off-manifold-collapse-plm

蛋白质设计语言模型生成质量后处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。