用CNN自编码器设计新注意力机制，提升多变量长期时间序列预测效率与精度。
CASA: CNN Autoencoder-based Score Attention for Efficient Multivariate Long-term Time-series Forecasting
- 基于CNN自编码器设计分数注意力，减少内存占用。
- 在8个真实数据集上推理速度提升44.0%，资源消耗降低77.7%。
- 适配多种Transformer模型，适合追求高效高精度预测的场景。
多变量长期时间序列预测在天气预报、交通分析等应用中至关重要。尽管基于Transformer的变体已显著提升预测精度，但现有方法仍存在时间复杂度高、计算资源消耗大及跨维度交互不足的问题。为此，本文提出一种新型的基于CNN自编码器的分数注意力机制（CASA），可无模型依赖地集成到各类Transformer中，有效降低内存开销并提升性能。在8个真实世界数据集上的实验表明，CASA将计算资源消耗最高降低77.7%，推理速度提升44.0%，并在87.5%的评估指标中达到最优表现，优于现有方法。
原文摘要 · Abstract (English)
Multivariate long-term time series forecasting is critical for applications such as weather prediction, and traffic analysis. In addition, the implementation of Transformer variants has improved prediction accuracy. Following these variants, different input data process approaches also enhanced the field, such as tokenization techniques including point-wise, channel-wise, and patch-wise tokenization. However, previous studies still have limitations in time complexity, computational resources, and cross-dimensional interactions. To address these limitations, we introduce a novel CNN Autoencoder-based Score Attention mechanism (CASA), which can be introduced in diverse Transformers model-agnosticically by reducing memory and leading to improvement in model performance. Experiments on eight real-world datasets validate that CASA decreases computational resources by up to 77.7%, accelerates inference by 44.0%, and achieves state-of-the-art performance, ranking first in 87.5% of evaluated metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。