用变元嵌入提升多变量时间序列相关性建模能力
VE: Modeling Multivariate Time Series Correlation with Variate Embedding
- 为每个变量学习唯一嵌入，结合MoE与LoRA控制参数量
- 在4个数据集上显著提升预测性能，有效分组相似时序变量
- 可插拔集成到现有模型，适合多变量时间序列研究者
多变量时间序列预测依赖于准确捕捉变量间的相关性。当前的通道独立（CI）模型及带有CI最终投影层的模型无法有效建模这些依赖关系。本文提出变元嵌入（VE）管道，为每个变量学习唯一且一致的嵌入，并结合混合专家（MoE）与低秩适配（LoRA）技术，在控制参数规模的同时提升预测性能。该方法可无缝集成至任何具有CI最终投影层的模型中，用于改进多变量时间序列预测。实验表明，所学的VE能有效将具有相似时间模式的变量聚类，分离低相关性变量。在四个常用数据集上的实验验证了其有效性。代码已开源：https://github.com/swang-song/VE。
原文摘要 · Abstract (English)
Multivariate time series forecasting relies on accurately capturing the correlations among variates. Current channel-independent (CI) models and models with a CI final projection layer are unable to capture these dependencies. In this paper, we present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate and combines it with Mixture of Experts (MoE) and Low-Rank Adaptation (LoRA) techniques to enhance forecasting performance while controlling parameter size. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting. The learned VE effectively groups variates with similar temporal patterns and separates those with low correlations. The effectiveness of the VE pipeline is demonstrated through experiments on four widely-used datasets. The code is available at: https://github.com/swang-song/VE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。