轻量级双向基因组模型，提升植物基因序列理解能力
PlantBiMoE: A Bidirectional Foundation Model with SparseMoE for Plant Genomes
- 结合双向Mamba与稀疏专家混合架构，捕捉正反链结构依赖
- 在31个任务中20项表现最佳，平均性能领先现有模型
- 适合植物基因组分析、基因编辑与合成生物学研究者使用
理解植物基因组的潜在语言规则是计算生物学中的基础挑战。尽管AgroNT和PDLLMs等模型取得进展，但仍存在参数量过大和难以建模DNA双链双向性的问题。为此，我们提出PlantBiMoE，一种轻量且表达能力强的植物基因组语言模型，融合双向Mamba与稀疏专家混合（SparseMoE）框架。双向Mamba有效捕捉正链与反链间的结构依赖，SparseMoE显著减少活跃参数数量，提升计算效率而不损失建模能力。我们在增强版基因组基准MPGB上评估模型，该基准整合了11个代表性任务的31个数据集，输入序列长度为50至6,000 bp。实验结果表明，PlantBiMoE在31个数据集中有20项表现最优，平均性能亦达最佳。综上，该模型能有效表征植物基因序列，为多种基因组任务提供可靠计算工具，对植物基因组学、基因编辑与合成生物学具有重要贡献。代码已开源：https://github.com/HUST-Keep-Lin/PlantBiMoE
原文摘要 · Abstract (English)
Understanding the underlying linguistic rules of plant genomes remains a fundamental challenge in computational biology. Recent advances including AgroNT and PDLLMs have made notable progress although, they suffer from excessive parameter size and limited ability to model the bidirectional nature of DNA strands respectively. To address these limitations, we propose PlantBiMoE, a lightweight and expressive plant genome language model that integrates bidirectional Mamba and a Sparse Mixture-of-Experts (SparseMoE) framework. The bidirectional Mamba enables the model to effectively capture structural dependencies across both the forward and reverse DNA strands, while SparseMoE significantly reduces the number of active parameters, improving computational efficiency without sacrificing modeling capacity. We evaluated and tested our model on the Modified Plants Genome Benchmark (MPGB), an enhanced genomic benchmark, which consolidates 31 datasets across 11 representative tasks, with input sequence lengths ranging from 50 to 6,000 bp. Experimental results demonstrate that PlantBiMoE achieves the best performance on 20 out of 31 datasets and the average best when comparing with existing models. In summary, all above results demonstrate that our model can effectively represent plant genomic sequences, serving as a robust computational tool for diverse genomic tasks, while making substantive contributions to plant genomics, gene editing, and synthetic biology. The code is available at: https://github.com/HUST-Keep-Lin/PlantBiMoE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。