Kairos用动态分块和自适应编码,让时间序列模型更省参数、泛化更强。
Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Models
- 动态分块 tokenizer 根据信息密度调整粒度,避免固定编码带来的记忆偏差。
- 在 GIFT-Eval 和 Time-Series-Library 上零样本性能领先,参数量却显著减少。
- 适合追求高效、可扩展时间序列建模的研究者与工业应用开发者。
时间序列固有的时序异质性,如采样密度差异和周期结构变化,严重制约了时间序列基础模型(TSFMs)的零样本泛化能力。现有模型多依赖大规模参数来吸收这种异质性,因其静态分块和位置编码将不同时间模式纠缠于固定表示空间,导致模型更倾向记忆而非适应。为此,我们提出 Kairos,一种灵活且参数高效的 TSFM,专用于预测任务,通过新颖的分块视角将时间异质性与模型容量解耦。Kairos引入动态分块分词器和大小混合编码,根据局部信息密度自适应调整观测粒度,实现细粒度时序抽象而不增加模型宽度或深度。此外,设计基于动态旋转编码的多粒度位置嵌入,以实例级频谱特征和动态分块诱导的时序结构为条件,实现对多样化时序依赖的稳健建模。在新型可预测性分层时间序列(PreSTS)语料库上训练,Kairos在两个主流基准 GIFT-Eval 与 Time-Series-Library 上均取得优异零样本表现,且参数量远低于现有方法。
原文摘要 · Abstract (English)
Inherent temporal heterogeneity, such as varying sampling densities and periodic structures, has posed substantial challenges in zero-shot generalization for Time Series Foundation Models (TSFMs). Existing TSFMs predominantly rely on massive parameterization to absorb such heterogeneity, as their static tokenization and positional encoding schemes entangle diverse temporal patterns into a fixed representation space, encouraging memorization rather than adaptation. To address this limitation, we propose Kairos, a flexible and parameter-efficient TSFM dedicated to forecasting tasks, which decouples temporal heterogeneity from model capacity through a novel tokenization perspective. Kairos introduces a dynamic patching tokenizer and a mixture-of-size encoding that adapt observational granularity to local information density, enabling fine-grained temporal abstraction without increasing model width or depth. In addition, we design a multi-granularity positional embedding based on dynamic rotary encodings, which conditions on instance-level spectral features and temporal structure induced by dynamic patching tokenization, allowing robust modeling of diverse temporal dependencies. Trained on a novel Predictability-Stratified Time-Series (PreSTS) corpus, Kairos achieves superior zero-shot performance with substantially fewer parameters on two mainstream benchmarks, GIFT-Eval and Time-Series-Library. The project page is at https://foundation-model-research.github.io/Kairos .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。