让时间序列模型处理更长的多变量数据，提升医疗等领域的应用能力。
Towards Long-Context Time Series Foundation Models
- 提出压缩记忆机制，增强编码器模型对多变量依赖关系的建模能力。
- 在多任务时间序列基础模型MOMENT上实现长上下文支持，性能显著提升。
- 系统对比语言与时间序列领域上下文扩展技术，为后续研究提供参考。
时间序列基础模型在多个领域表现出色，甚至在零样本设置下也表现优异。然而，大多数模型仅能处理短时单变量时间序列，限制了其在医疗等领域中大量长时多变量数据的应用。本文系统梳理并比较了来自语言和时间序列领域的多种上下文扩展技术，提出一种新型压缩记忆机制，使仅含编码器的时间序列基础模型（TSFM）能有效建模变量内部依赖关系。通过将该机制融入近期提出的多任务时间序列基础模型MOMENT，验证了其在长上下文场景下的有效性。
原文摘要 · Abstract (English)
Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. However, most of these models are designed to handle short univariate time series as an input. This limits their practical use, especially in domains such as healthcare with copious amounts of long and multivariate data with strong temporal and intra-variate dependencies. Our study bridges this gap by cataloging and systematically comparing various context expansion techniques from both language and time series domains, and introducing a novel compressive memory mechanism to allow encoder-only TSFMs to effectively model intra-variate dependencies. We demonstrate the benefits of our approach by imbuing MOMENT, a recent family of multi-task time series foundation models, with the multivariate context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。