将Transformer改进为适合连续时间序列预测的轻量架构
Minimal Time Series Transformer
- 仅做最小改动适配连续值时间序列数据
- 保持原Transformer自回归特性，有效融合历史信息
- 适合需要高效建模时序依赖的场景
Transformer是自然语言处理、计算机视觉和音频分析等领域的前沿模型，通过自回归方式将过去输入与输出样本的信息有效结合，使每个样本能感知全部历史。在序列到序列（Seq2Seq）建模中，这种机制显著提升对下一个输出的预测能力。时间序列预测本质上属于Seq2Seq问题。然而原始Transformer针对离散序列设计，需改造以适应连续值时间序列。本文提出仅需最少调整，即可让原生Transformer架构有效处理连续时间序列数据。
原文摘要 · Abstract (English)
Transformer is the state-of-the-art model for many natural language processing, computer vision, and audio analysis problems. Transformer effectively combines information from the past input and output samples in auto-regressive manner so that each sample becomes aware of all inputs and outputs. In sequence-to-sequence (Seq2Seq) modeling, the transformer processed samples become effective in predicting the next output. Time series forecasting is a Seq2Seq problem. The original architecture is defined for discrete input and output sequence tokens, but to adopt it for time series, the model must be adapted for continuous data. This work introduces minimal adaptations to make the original transformer architecture suitable for continuous value time series data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。