DNA语言模型用可学习分段方法，自动识别功能区与重复区。
DNACHUNKER: Learnable Tokenization for DNA Language Models
- 引入可学习的动态分段模块，按上下文生成长短不一的序列单元
- 在五个基准上优于固定分词基线，尤其在变异和重复区域表现更好
- 适合基因组功能分析、进化研究等生物信息学场景
DNA语言模型广泛用于基因组序列表征，但其性能高度依赖原始核苷酸到模型输入的转换方式。与自然语言不同，DNA无天然边界，固定分词在突变、插入缺失和局部重复下易失效。我们提出DNAChunker，一种带可学习自适应分段模块的掩码语言模型，能生成上下文相关的可变长度单元。基于动态分段机制，DNAChunker学会在功能富集区域使用更细粒度划分，同时压缩重复或冗余序列。我们在人类参考基因组上预训练该模型，并在五个基准上评估，结果持续优于强基线。进一步分析与消融实验表明，其分段方式具有生物学意义且对突变具备鲁棒性。
原文摘要 · Abstract (English)
DNA language models are increasingly used to represent genomic sequence, yet their effectiveness depends critically on how raw nucleotides are converted into model inputs. Unlike natural language, DNA offers no canonical boundaries, making fixed tokenizations a brittle design choice under shifts, indels, and local repeats. We introduce DNAChunker, a masked DNA language model that incorporates a learnable adaptive segmentation module to produce context-dependent, variable-length units. Building on a dynamic segmentation procedure, DNAChunker learns to allocate finer granularity to functionally enriched regions while compressing repetitive or redundant sequence. We pretrain DNAChunker on the human reference genome and evaluate it across five benchmarks, where it consistently improves over strong fixed-tokenization baselines. Further analyses and ablations indicate that unlike fixed tokenizations, segmentation is learned in a biologically-informed, mutation-resilient manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。