arXiv:2508.06641cs.LGcs.AI2025-08被引 1

用数学映射实现序列多尺度编码,让基因组数据高效嵌入数值空间。

Fractal Language Modelling by Universal Sequence Maps (USM)

  • 通过双向混沌游戏映射构建可逆的分形编码,保留符号序列上下文。
  • 解决初始种子偏差问题,使编码结果与序列身份完全一致。
  • 编码过程收敛于稳定状态,适用于任意字母表规模的序列建模。

随着Transformer语言模型(如ChatGPT)的兴起,人们重新关注能以多尺度和多维度数值表示符号序列的编码方法。核心挑战在于如何唯一保留符号连续性带来的上下文信息,以便由神经网络等非线性模型建模。本文提出通用序列映射(USM),一种基于两个混沌游戏表示(CGR)正向与反向迭代的双射编码方法,可投影至频域形成FCGR。其坐标可用于计算切比雪夫距离与k-mer频率,无需重复计算嵌入坐标,并且能支持非整数的k值。本研究解决了影响迭代过程的种子偏差问题,带来两个结果:一是实现了数值位置与序列身份的完全一致;二是揭示了USM本质上是一种高效收敛至稳定嵌入解的数值过程。实验以仅含4个字符的基因组序列(4种核苷酸)为例,因其可在平面中直观表示,但研究表明该方法可直接扩展至任意大小字母表。

原文摘要 · Abstract (English)

Motivation: With the advent of Language Models using Transformers, popularized by ChatGPT, there is a renewed interest in exploring encoding procedures that numerically represent symbolic sequences at multiple scales and embedding dimensions. The challenge that encoding addresses is the need for mechanisms that uniquely retain contextual information about the succession of individual symbols, which can then be modeled by nonlinear formulations such as neural networks. Context: Universal Sequence Maps(USM) are iterated functions that bijectively encode symbolic sequences onto embedded numerical spaces. USM is composed of two Chaos Game Representations (CGR), iterated forwardly and backwardly, that can be projected into the frequency domain (FCGR). The corresponding USM coordinates can be used to compute a Chebyshev distance metric as well as k-mer frequencies, without having to recompute the embedded numeric coordinates, and, paradoxically, allowing for non-integers values of k. Results: This report advances the bijective fractal encoding by Universal Sequence Maps (USM) by resolving seeding biases affecting the iterated process. The resolution had two results, the first expected, the second an intriguing outcome: 1) full reconciliation of numeric positioning with sequence identity; and 2) uncovering the nature of USM as an efficient numeric process converging towards a steady state sequence embedding solution. We illustrate these results for genomic sequences because of the convenience of a planar representation defined by an alphabet with only 4 tokens (the 4 nucleotides). Nevertheless, the application to alphabet of arbitrary cardinality was found to be straightforward.

序列编码分形映射基因组建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。