Helix-mRNA可高效预测mRNA全序列功能,兼顾编码区与非翻译区。
Helix-mRNA: A Hybrid Foundation Model For Full Sequence mRNA Therapeutics
- 基于状态空间与注意力混合架构,用单核苷酸分词保留生物结构信息
- 处理长度是现有方法6倍,参数量仅需10%,全面预测mRNA各区域特性
- 适合药物研发人员优化mRNA疫苗序列,开源可用
基于mRNA的疫苗已成为制药业重点。mRNA的编码序列及非翻译区(UTRs)会显著影响翻译效率、稳定性、降解等,共同决定疫苗效果。然而,优化mRNA序列以提升这些性质仍具挑战性。现有深度学习模型通常仅关注编码区,忽视了UTRs。我们提出Helix-mRNA,一种基于结构化状态空间与注意力机制的混合模型。在首次预训练后,通过高质量数据进行二次预训练以实现专业化。采用单核苷酸分词并保留密码子分隔,确保原始序列中的生物与结构信息不丢失。Helix-mRNA在分析编码区和非翻译区特性方面优于现有方法,可处理长达6倍于当前方法的序列,同时仅需现有基础模型10%的参数量。其预测能力覆盖所有mRNA区域。模型已开源(https://github.com/helicalAI/helical),权重发布于Hugging Face(https://huggingface.co/helical-ai/helix-mRNA)。
原文摘要 · Abstract (English)
mRNA-based vaccines have become a major focus in the pharmaceutical industry. The coding sequence as well as the Untranslated Regions (UTRs) of an mRNA can strongly influence translation efficiency, stability, degradation, and other factors that collectively determine a vaccine's effectiveness. However, optimizing mRNA sequences for those properties remains a complex challenge. Existing deep learning models often focus solely on coding region optimization, overlooking the UTRs. We present Helix-mRNA, a structured state-space-based and attention hybrid model to address these challenges. In addition to a first pre-training, a second pre-training stage allows us to specialise the model with high-quality data. We employ single nucleotide tokenization of mRNA sequences with codon separation, ensuring prior biological and structural information from the original mRNA sequence is not lost. Our model, Helix-mRNA, outperforms existing methods in analysing both UTRs and coding region properties. It can process sequences 6x longer than current approaches while using only 10% of the parameters of existing foundation models. Its predictive capabilities extend to all mRNA regions. We open-source the model (https://github.com/helicalAI/helical) and model weights (https://huggingface.co/helical-ai/helix-mRNA).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。