用基因组特征监督训练，让DNA模型学得更懂功能。
SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model
- 用基因组特征预测任务替代纯序列预训练
- 跨物种多特征任务上达到顶尖性能
- 适合作为生物信息学研究的通用基础模型
受无监督预训练成功的启发，研究者已尝试将此类方法应用于DNA预训练。然而我们认为，仅依赖纯DNA序列会导致结果不理想，因为其功能由染色质可及性等基因组特征调控。本文证明，以基因组特征预测为目标的有监督训练比纯序列预训练更有效。针对基因组特征预测的多物种、多特征特性，我们提出SPACE（Species-Profile Adaptive Collaborative Experts），采用专家混合（MoE）机制，更好地捕捉不同物种和基因组特征间的关联，从而学习更有效的DNA表示。在多个任务上的广泛实验表明，该模型达到当前最优表现，证实了基于基因组特征训练的DNA模型是强大的表示学习器。代码已开源：https://github.com/ZhuJiwei111/SPACE。
原文摘要 · Abstract (English)
Inspired by the success of unsupervised pre-training paradigms, researchers have applied these approaches to DNA pre-training. However, we argue that these approaches alone yield suboptimal results because pure DNA sequences lack sufficient information, since their functions are regulated by genomic profiles like chromatin accessibility. Here, we demonstrate that supervised training for genomic profile prediction serves as a more effective alternative to pure sequence pre-training. Furthermore, considering the multi-species and multi-profile nature of genomic profile prediction, we introduce our $\textbf{S}$pecies-$\textbf{P}$rofile $\textbf{A}$daptive $\textbf{C}$ollaborative $\textbf{E}$xperts (SPACE) that leverages Mixture of Experts (MoE) to better capture the relationships between DNA sequences across different species and genomic profiles, thereby learning more effective DNA representations. Through extensive experiments across various tasks, our model achieves state-of-the-art performance, establishing that DNA models trained with supervised genomic profiles serve as powerful DNA representation learners. The code is available at https://github.com/ZhuJiwei111/SPACE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。