将时间序列视为语言,用概率分布建模实现跨领域迁移。
The language of time: a language model perspective on time-series foundation models
- 把时间序列切片转为离散词汇,类比语言模型表示方式。
- 实验证明时间序列片段可被忠实量化,统计特性与自然语言一致。
- 为时序大模型的泛化能力提供理论支撑,适合研究者参考。
随着大规模语言模型的兴起,基于海量参数和数据训练基础模型的范式在多个领域取得显著成功。时间序列基础模型是该范式的重大延伸,展现出卓越的表达能力、泛化性与跨领域迁移能力。然而,这引发了一个根本性悖论:时间序列反映不同的动态系统,跨领域迁移看似不可能,却与模型的实证成功相矛盾。本文从理论与实验双重视角,探究基于补丁的时间序列基础模型的表征学习机制与泛化能力。我们认为,这类模型并非仅引入新架构,而是从根本上将语言模型的表征范式推广至潜在的概率分布形式,将确定性向量表示扩展为概率分布表示。理论分析表明,连续时间序列补丁可被忠实量化为离散词汇,其关键统计特性高度契合自然语言。这一泛化使时间序列模型得以继承语言模型的鲁棒表征与迁移能力,从而解释其在时序任务中的优异表现。最终,本工作为理解、评估与改进大规模时间序列基础模型的安全性与可靠性提供了严格的理论基石。
原文摘要 · Abstract (English)
With the rise of large language models, the paradigm of training foundation models with massive parameter counts on vast datasets has been adopted in multiple domains to achieve remarkable success. Time series foundation models represent a significant extension of this paradigm, demonstrating exceptional expressive power, generalization, and cross-domain transferability. However, this gives rise to a fundamental paradox: time series data reflect distinct dynamical systems, making cross-domain transfer intuitively implausible, yet this is contradicted by the models' empirical success. To resolve this paradox, this paper investigates, from both theoretical and experimental perspectives, the representation learning mechanisms and generalization capabilities of patch-based time series foundation models. We argue that such models are not merely applying a new architecture but are fundamentally generalizing the representation paradigm of language models by extending deterministic vector-based representations to latent probabilistic distributional forms. Our theoretical analysis supports this framework by demonstrating that continuous time-series patches can be faithfully quantized into a discrete vocabulary whose key statistical properties are highly consistent with those of natural language. This generalization allows time series models to inherit the robust representation and transfer abilities of large language models, thereby explaining their superior performance in temporal tasks. Ultimately, our work provides a rigorous theoretical cornerstone for understanding, evaluating, and improving the safety and reliability of large-scale time series foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。