用CNN和Transformer融合预测时间序列,还能解释模型决策过程。
CNN-TFT explained by SHAP with multi-head attention weights for time series forecasting
- 先用CNN提取局部特征,再用TFT捕捉长短时依赖
- 在水电流量数据上误差低至2.2%,优于主流模型
- 结合SHAP与多头注意力权重,让预测结果可解释
卷积神经网络(CNN)和变换器架构在建模时序数据方面各具优势:CNN擅长捕捉局部模式和平移不变性,而变换器通过自注意力机制有效建模长程依赖。本文提出一种混合架构,将卷积特征提取与时间融合变换器(TFT)主干结合,以提升多变量时间序列预测性能。CNN模块首先应用一系列一维卷积层,从原始输入序列中提炼出显著的局部模式,降低噪声和维度。生成的特征图随后输入TFT,利用多头注意力机制捕捉短期与长期依赖,并自适应加权相关协变量。我们在水力发电自然流量时间序列数据集上评估了CNN-TFT模型。实验结果表明,该模型优于多个成熟的深度学习模型,平均绝对百分比误差(MAPE)最高可达2.2%。模型的可解释性通过提出的基于多头注意力权重的加性解释方法(SHAP-MHAW)实现。我们提出的新型架构CNN-TFT-SHAP-MHAW,适用于对高保真度、多变量时间序列预测有需求的应用场景,代码已开源:https://github.com/SFStefenon/CNN-TFT-SHAP-MHAW。
原文摘要 · Abstract (English)
Convolutional neural networks (CNNs) and transformer architectures offer strengths for modeling temporal data: CNNs excel at capturing local patterns and translational invariances, while transformers effectively model long-range dependencies via self-attention. This paper proposes a hybrid architecture integrating convolutional feature extraction with a temporal fusion transformer (TFT) backbone to enhance multivariate time series forecasting. The CNN module first applies a hierarchy of one-dimensional convolutional layers to distill salient local patterns from raw input sequences, reducing noise and dimensionality. The resulting feature maps are then fed into the TFT, which applies multi-head attention to capture both short- and long-term dependencies and to weigh relevant covariates adaptively. We evaluate the CNN-TFT on a hydroelectric natural flow time series dataset. Experimental results demonstrate that CNN-TFT outperforms well-established deep learning models, with a mean absolute percentage error of up to 2.2%. The explainability of the model is obtained by a proposed Shapley additive explanations with multi-head attention weights (SHAP-MHAW). Our novel architecture, named CNN-TFT-SHAP-MHAW, is promising for applications requiring high-fidelity, multivariate time series forecasts, being available for future analysis at https://github.com/SFStefenon/CNN-TFT-SHAP-MHAW .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。