用元数据提升时间序列预测,让模型理解场景背景。
Metadata Matters for Time Series: Informative Forecasting with Transformers
- 将元数据转为自然语言,用大模型编码成补充嵌入。
- 在多个基准上优于现有模型,长短期预测均表现优异。
- 适合需要解释性、跨场景泛化的预测任务。
时间序列预测广泛应用于金融分析和能源规划等实际场景。以往研究主要关注时间序列本身的特征,试图捕捉其复杂变化与依赖关系。然而,除了数值数据外,元数据(如数据集描述、变量说明)也蕴含重要信息,有助于识别应用场景并提供比数字序列更可解释的知识。受此启发,我们提出元数据驱动的时间序列Transformer模型MetaTST,通过引入多层次上下文相关的元数据,增强Transformer预测模型的可解释性。针对元数据的非结构化特性,MetaTST采用预设模板将其转化为自然语言,并利用大语言模型(LLMs)编码为元数据标记,作为经典序列标记的补充,形成富有信息量的嵌入表示。进一步,使用Transformer编码器实现序列与元数据标记间的交互,使序列表征能融合元数据信息,提升预测精度。该设计还能自适应学习不同场景下的特定模式,特别适用于大规模、多场景的预测任务。实验表明,MetaTST在多个公认的短时与长时预测基准上超越先进时间序列模型及基于LLM的方法,涵盖单数据集独立训练与多数据集联合训练两种设置。
原文摘要 · Abstract (English)
Time series forecasting is prevalent in extensive real-world applications, such as financial analysis and energy planning. Previous studies primarily focus on time series modality, endeavoring to capture the intricate variations and dependencies inherent in time series. Beyond numerical time series data, we notice that metadata (e.g.~dataset and variate descriptions) also carries valuable information essential for forecasting, which can be used to identify the application scenario and provide more interpretable knowledge than digit sequences. Inspired by this observation, we propose a Metadata-informed Time Series Transformer (MetaTST), which incorporates multiple levels of context-specific metadata into Transformer forecasting models to enable informative time series forecasting. To tackle the unstructured nature of metadata, MetaTST formalizes them into natural languages by pre-designed templates and leverages large language models (LLMs) to encode these texts into metadata tokens as a supplement to classic series tokens, resulting in an informative embedding. Further, a Transformer encoder is employed to communicate series and metadata tokens, which can extend series representations by metadata information for more accurate forecasting. This design also allows the model to adaptively learn context-specific patterns across various scenarios, which is particularly effective in handling large-scale, diverse-scenario forecasting tasks. Experimentally, MetaTST achieves state-of-the-art compared to advanced time series models and LLM-based methods on widely acknowledged short- and long-term forecasting benchmarks, covering both single-dataset individual and multi-dataset joint training settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。