dnaHNet无需分词，端到端建模基因组序列，效率提升3倍以上。
dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning
- 采用可微分动态分块机制，自适应压缩核苷酸为隐变量
- 相比Transformer减少80%计算量，推理速度提升3倍以上
- 零样本预测蛋白变异适应度和基因必需性表现优异
基因组基础模型有望解码DNA语法，但其输入表示面临根本权衡：标准固定词汇分词器会破坏密码子和调控元件等生物意义片段，而核苷酸级模型虽保持生物学一致性，却在长序列上带来难以承受的计算成本。我们提出dnaHNet，一种免分词的自回归模型，可端到端地对基因组序列进行分段与建模。通过可微分动态分块机制，dnaHNet自适应地将原始核苷酸压缩为隐向量，平衡压缩率与预测精度。在原核生物基因组上预训练后，dnaHNet在扩展性和效率方面优于StrippedHyena2等先进架构。该递归分块策略实现二次方级浮点运算量降低，使推理速度相较Transformer提升3倍以上。在零样本任务中，dnaHNet在预测蛋白变异适应度和基因必需性方面表现更优，且无需监督即可自动发现层级化生物结构。这些结果确立了dnaHNet作为下一代基因组建模的可扩展、可解释框架。
原文摘要 · Abstract (English)
Genomic foundation models have the potential to decode DNA syntax, yet face a fundamental tradeoff in their input representation. Standard fixed-vocabulary tokenizers fragment biologically meaningful motifs such as codons and regulatory elements, while nucleotide-level models preserve biological coherence but incur prohibitive computational costs for long contexts. We introduce dnaHNet, a state-of-the-art tokenizer-free autoregressive model that segments and models genomic sequences end-to-end. Using a differentiable dynamic chunking mechanism, dnaHNet compresses raw nucleotides into latent tokens adaptively, balancing compression with predictive accuracy. Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency. This recursive chunking yields quadratic FLOP reductions, enabling $>3 \times$ inference speedup over Transformers. On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision. These results establish dnaHNet as a scalable, interpretable framework for next-generation genomic modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。