通过改进嵌入方式，让时间序列模型更好捕捉变量间关系。
Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding
- 在嵌入层加入可学习位置编码与轻量路由注意力，实现跨变量信息注入。
- 在7个真实数据集上，性能全面超越原始模型，提升显著。
- 适合需要提升多变量依赖建模能力的研究者和工程师使用。
Transformer 在时间序列预测中因能捕捉长期依赖而受到欢迎。然而，许多现有模型仅关注时间依赖性，忽略了变量间的复杂关系。近期模型尝试通过顺序或统一注意力机制同时建模跨时间与跨变量依赖，但所有层均依赖通道（CD），易导致过拟合。为此，本文提出交叉变量补丁嵌入（CVPE），一种轻量级的通道独立（CI）模块，通过修改补丁嵌入过程，将跨变量上下文注入到通道独立模型中。具体方法是在原始补丁嵌入层中加入可学习的位置编码与轻量级路由注意力块。将 CVPE 集成到 Time-LLM（一种多模态通道独立预测模型）中，验证其在捕捉跨变量依赖方面的有效性，并提升模型性能。在七个真实世界数据集上的大量实验表明，仅通过引入 CVPE 模块，增强版 Time-LLM 即可显著优于原基线模型，且无需其他改动。
原文摘要 · Abstract (English)
Transformers have recently gained popularity in time series forecasting due to their ability to capture long-term dependencies. However, many existing models focus only on capturing temporal dependencies while omitting intricate relationships between variables. Recent models have tried tackling this by explicitly modeling both cross-time and cross-variate dependencies through a sequential or unified attention mechanism, but they are entirely channel dependent (CD) across all layers, making them potentially susceptible to overfitting. To address this, we propose Cross-Variate Patch Embeddings (CVPE), a lightweight CD module that injects cross-variate context into channel-independent (CI) models by simply modifying the patch embedding process. We achieve this by adding a learnable positional encoding and a lightweight router-attention block to the vanilla patch embedding layer. We then integrate CVPE into Time-LLM, a multimodal CI forecasting model, to demonstrate its effectiveness in capturing cross-variate dependencies and enhance the CI model's performance. Extensive experimental results on seven real-world datasets show that our enhanced Time-LLM outperforms the original baseline model simply by incorporating the CVPE module, with no other changes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。