arXiv:2606.18703cs.LGq-bio.QM2026-06

让生物语言模型根据上下文精准预测突变影响,提升药物耐药性等任务性能。

Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment

论文配图:Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment
图 1 · 摘自论文原文
  • 在输出概率空间直接做对比学习,保留原始模型的词元概率接口。
  • 在药物耐药性任务中将AUC从0.55提升至0.65,优于现有方法。
  • 适用于不同模型间跨模态匹配,无需共享分词器或解码器。

预训练生物语言模型通过掩码词预测暴露每个词元的概率分布,构成了序列设计、变异评分和机制解释的核心接口。然而这些分布来自广泛无标注语料,未自然关联任务相关的生物上下文(如相互作用伙伴、细胞环境或治疗干预)。现有上下文匹配方法常通过聚合嵌入、对比潜在空间或任务特异性预测头扭曲该接口。我们提出LOGICA(Logit-space Contrastive Alignment)框架,实现直接在输出对数概率空间进行上下文条件化预测的对比学习。通过与各模型原生词元头兼容的门控跨模态适配器,LOGICA保持预训练概率接口,并将上下文化的词元对数似然转化为匹配得分。对齐基于上下文敏感的词元概率,而非共享嵌入空间中的距离,支持在模型词汇表不同的情况下,仅用稀疏成对数据进行学习,无需共享分词器或解码器。LOGICA在突变局部变异排序任务中尤为有效,比较简化为扰动位点上突变词元的上下文条件似然。在蛋白-配体结合、TCR-肽活性及药物诱导耐药性预测任务中,其性能优于先前最优方法,包括匹配的潜在对比和条件掩码语言建模基线,同时保留词元级接口以支持解释与生成。在独立基因单突变耐药性预测任务中,其AUC从近随机的0.55提升至约0.65。

原文摘要 · Abstract (English)

Pretrained biological language models expose per-token probability distributions through masked-token prediction, providing the likelihood interface central to sequence design, variant scoring, and mechanistic interpretation. Yet these distributions are learned from broad unlabeled corpora and are not naturally conditioned on task-specific biological contexts such as interaction partners, cellular environments, or therapeutic interventions. Existing contextual matching methods often distort this interface through pooled embeddings, contrastive latent spaces, or task-specific prediction heads. We introduce LOGICA (Logit-space Contrastive Alignment), a framework for context-conditioned prediction that performs contrastive learning directly in output-logit space. Using gated cross-modal adapters compatible with each model's native token head, LOGICA preserves the pretrained likelihood interface and converts contextualized token log-likelihoods into matching scores. Alignment is defined through context-sensitive token probabilities rather than proximity in a shared embedding space, enabling learning from sparse paired data across models with distinct vocabularies, without a shared tokenizer or decoder. LOGICA is particularly effective for mutation-local variant ranking, where comparisons reduce to context-conditioned likelihoods of mutant tokens at perturbed sites. Across protein--ligand binding, TCR--peptide activity, and drug-conditioned resistance prediction, LOGICA improves over prior state-of-the-art methods, including matched latent-contrastive and conditional MLM baselines, while retaining a token-level interface for interpretation and generation. On held-out-gene single-mutation drug-resistance prediction, LOGICA improves AUC from near-random latent-space baselines of $\sim$0.55 to $\sim$0.65.

生物语言模型对比学习变异预测药物耐药性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。