arXiv:2510.08169cs.LG2025-10NeurIPS被引 1

用双向特征增强自回归模型,提升肽段生成质量

Bidirectional Representations Augmented Autoregressive Biological Sequence Generation

  • 将非自回归模型的双向上下文融入自回归生成过程
  • 在九物种去新肽段测序任务中显著超越基线模型
  • 适合需要高精度序列生成的生物信息学研究者

自回归(AR)模型在肽段去新测序和蛋白质建模等生物任务中受限于单向性,难以捕捉关键的全局双向依赖。非自回归(NAR)模型虽具整体双向表示能力,但存在生成连贯性差与可扩展性不足的问题。为此,我们提出一种混合框架,通过动态整合NAR机制的丰富上下文信息来增强AR生成。该方法共享一个输入编码器,搭配两个解码器:一个NAR解码器学习潜在的双向生物特征,一个AR解码器利用这些特征逐步合成序列。创新的跨解码器注意力模块使AR解码器可迭代查询并融合双向特征,提升预测质量。训练采用重要性退火策略实现目标平衡,并通过跨解码器梯度阻断保证学习稳定。在涵盖九个物种的去新肽段测序基准测试中,本模型显著优于AR和NAR基线。其成功融合了AR的稳定性与NAR的上下文感知能力,在多样下游数据上表现卓越。该研究推进了生物序列建模技术,提出了增强AR模型双向理解的新架构。代码已开源:https://github.com/BEAM-Labs/denovo。

原文摘要 · Abstract (English)

Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks such as de novo peptide sequencing and protein modeling by their unidirectional nature, failing to capture crucial global bidirectional token dependencies. Non-Autoregressive (NAR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability. To transcend this, we propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared input encoder with two decoders: a non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesizing the biological sequence by leveraging these bidirectional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross-decoder gradient blocking for stable, focused learning. Evaluations on a demanding nine-species benchmark of de novo peptide sequencing show that our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, superior performance on diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation. Code is available at https://github.com/BEAM-Labs/denovo.

序列生成生物信息自回归双向建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。