arXiv:2606.09861cs.LGcs.AI2026-06

将时间序列转为离散符号,用通用模型实现零样本预测与跨任务推理。

Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models

论文配图:Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models
图 1 · 摘自论文原文
  • 设计向量量化自编码器,将连续时间序列转为离散令牌。
  • 单个模型在多任务上表现优于传统方法,达任务专用模型水平。
  • 无需训练即可通过上下文提示完成预测、生成和分类,适合通用时序建模。

尽管下一词预测(NTP)已统一大语言模型预训练范式,但其在无界连续时间序列(TS)上的适配仍待解决。为此,我们提出UniTok,一种将时间序列转换为离散令牌的通用分词器,以及基于此令牌预训练的UniTok-FM基础模型。UniTok-FM是通用型基础模型,支持零样本与提示增强的预测,以及无需训练的上下文内推理,在生成与分类任务中表现优异——这是以往工作未能实现的能力。技术上,UniTok采用向量量化自编码器,结合前缀归一化稳定尺度,渐进式分辨率因果架构进行编码解码,并使用结构保持重建损失训练。UniTok-FM采用现成的大语言模型架构,无时序特定修改。预训练不依赖孤立时间序列,而是对具有相似模式的多序列构成的上下文窗口执行NTP,以捕捉其共享动态。在预测、生成与分类任务上的实验表明,单一统一的UniTok-FM持续优于统计与监督基线,性能媲美任务专用基础模型,并唯一实现了跨任务的训练自由上下文内推理。

原文摘要 · Abstract (English)

While Next-Token Prediction (NTP) has unified LLM pretraining, its adaptation to unbounded, continuous time series (TS) remains open. To bridge the gap, we introduce UniTok, a universal tokenizer that transforms TS into discrete tokens, and UniTok-FM, a foundation model pretrained via NTP on these tokens. UniTok-FM is a general-purpose foundation model that supports zero-shot and prompt-boosted forecasting, as well as few-shot generation and classification via training-free in-context inference--a capability not achieved by prior works. Technically, UniTok is a vector-quantized autoencoder incorporating prefix normalization for scale stabilization, a progressive-resolution causal architecture for encoding and decoding, and a structure-preserving reconstruction loss for training. UniTok-FM adopts an off-the-shelf LLM architecture without TS-specific modifications. Instead of pretraining on isolated TS, it performs NTP on context windows formed by multiple series with similar patterns, aiming to capture their shared dynamics. Experiments on forecasting, generation, and classification show that a single unified UniTok-FM consistently outperforms statistical and supervised baselines, achieves competitive performance with task-specific foundation models, and uniquely enables training-free in-context inference across tasks.

时间序列基础模型令牌化上下文推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。