arXiv:2508.04739q-bio.GNcs.LG2025-08被引 1

用轻量适配器让DNA模型直接分析RNA,省去预训练、少用80%参数。

CodonMoE: DNA Language Models for mRNA Analyses

  • 在DNA模型上加编码子专家混合模块,无需RNA数据重训。
  • 4项任务中表现优于原模型,参数比专用RNA模型少80%。
  • 适合想用现成DNA模型做RNA分析的研究者,降低算力门槛。

基因组语言模型(gLMs)面临效率瓶颈:要么为不同生物模态(如DNA和RNA)维护独立模型,要么构建大型多模态架构,两者都带来巨大计算开销。为此,我们提出CodonMoE(自适应编码子重构专家混合),一种轻量级适配器,可将DNA语言模型转化为有效的RNA分析工具,无需进行专门的RNA预训练。理论分析表明,当专家容量充足时,CodonMoE在密码子层面具有通用逼近能力,能将任意密码子序列映射到对应的RNA属性。在涵盖稳定性、表达水平和调控的四项RNA预测任务中,加入CodonMoE的DNA模型显著优于原始版本,HyenaDNA+CodonMoE系列仅用专用RNA模型80%的参数即达到最先进性能。该方法保持次二次复杂度,实现了更优性能与更低计算开销的统一,为整合基因组语言建模提供了可扩展路径,充分利用丰富的DNA数据,同时保留模态特异性优势。

原文摘要 · Abstract (English)

Genomic language models (gLMs) face a fundamental efficiency challenge: either maintain separate specialized models for each biological modality (DNA and RNA) or develop large multi-modal architectures. Both approaches impose significant computational burdens - modality-specific models require redundant infrastructure despite inherent biological connections, while multi-modal architectures demand massive parameter counts and extensive cross-modality pretraining. To address this limitation, we introduce CodonMoE (Adaptive Mixture of Codon Reformative Experts), a lightweight adapter that transforms DNA language models into effective RNA analyzers without RNA-specific pretraining. Our theoretical analysis establishes CodonMoE as a universal approximator at the codon level, capable of mapping arbitrary functions from codon sequences to RNA properties given sufficient expert capacity. Across four RNA prediction tasks spanning stability, expression, and regulation, DNA models augmented with CodonMoE significantly outperform their unmodified counterparts, with HyenaDNA+CodonMoE series achieving state-of-the-art results using 80% fewer parameters than specialized RNA models. By maintaining sub-quadratic complexity while achieving superior performance, our approach provides a principled path toward unifying genomic language modeling, leveraging more abundant DNA data and reducing computational overhead while preserving modality-specific performance advantages.

基因组模型RNA分析轻量化设计编码子

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。