新模型融合单核苷酸变异，更准确预测基因功能变化。
BMFM-DNA: A SNP-aware DNA foundation model to capture variant effects
- 用SNP特化表示法训练基础模型，显式编码基因变异信息。
- 在启动子检测等任务上，性能全面优于传统模型。
- 适合研究基因变异影响的生物信息学与遗传学工作者。
大型语言模型在自然语言处理中表现卓越，已被用于解析DNA的语言，其中核苷酸序列可视为“词语”。然而，基因组与自然语言本质不同，缺乏明确词汇和一致语法规则。尽管如DNABERT、GENA-LM等DNA语言模型在基因相关任务中达到较高水平，但它们无法有效捕捉序列变异下的生物学功能。为此，我们预训练了两个整合序列变异（特别是单核苷酸多态性，SNPs）的生物医学基础模型（BMFM），分别为：基于参考基因组生成变长序列及其反向互补序列的BMFM-DNA-REF，以及采用新型变异编码表示法的BMFM-DNA-SNP。实验表明，将序列变异纳入模型能显著提升下游微调任务的表现。为验证实用性，我们在DNABERT-2引入的启动子检测任务中测试了多种SNP补全策略。目前基准测试仍有限，未来需更全面评估。模型与代码已通过HuggingFace发布，开源地址：https://github.com/BiomedSciAI/biomed-multi-omic。
原文摘要 · Abstract (English)
Large language models (LLMs) trained on text demonstrated remarkable results on natural language processing (NLP) tasks. These models have been adapted to decipher the language of DNA, where sequences of nucleotides act as "words" that encode genomic functions. However, the genome differs fundamentally from natural language, as it lacks clearly defined words or a consistent grammar. Although DNA language models (DNALMs) such as DNABERT, GENA-LM have achieved high level of performance on genome-related biological tasks, these models do not encode biological functions in the presence of sequence variations. To address this problem, we pre-train foundation models that effectively integrate sequence variations, in particular Single Nucleotide Polymorphisms (SNPs), as they underlie important biological functions. Specifically, we use ModernBERT to pre-train two different Biomedical Foundation Models (BMFM), namely, BMFM-DNA-REF in which the model is trained with sequences of varying lengths along with their reverse complements derived from the reference genome and BMFM-DNA-SNP in which the model is trained with sequences created using a novel representation scheme that encodes sequence variations. Our findings indicate that integrating sequence variations into DNALMs helps capture the biological functions as seen in improvements on all fine-tuning tasks. To explore the model's practical utility, we experimented with various strategies for SNP imputation on promoter detection task introduced in DNABERT-2. However, we acknowledge that the current benchmarks are limited in their ability to fully evaluate these models. To enable more comprehensive assessment in the future and encourage community contributions, we release our models through HuggingFace and the code to reproduce the results at https://github.com/BiomedSciAI/biomed-multi-omic
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。