arXiv:2410.14107cs.LG2024-10被引 34

用迁移学习提升变压器模型预测建筑能耗,实测效果与特征空间密切相关。

Transfer Learning on Transformers for Building Energy Consumption Forecasting -- A Comparative Study

  • 对比六种数据导向的迁移学习策略,适配不同特征空间。
  • 在无目标数据时迁移学习显著提升预测精度,尤其对气象特征敏感。
  • PatchTST模型优于原生Transformer和Informer,适合时间序列预测。

本研究探究了将迁移学习(TL)应用于Transformer架构以提升建筑能耗预测性能。尽管迁移学习已有研究,但以往工作多聚焦单一数据驱动策略或使用较旧模型(如RNN、CNN)。本文针对六种不同的数据导向迁移学习策略进行系统性实证分析,并考察其在不同特征空间下的表现。实验采用来自建筑数据基因组计划2的16个数据集,构建建筑能耗预测模型。除基础Transformer外,还测试了专为时间序列设计的Informer和PatchTST。结果表明,迁移学习总体有益,尤其在目标域无数据时;但最佳策略取决于特征空间特性,如气象数据记录情况。此外,PatchTST在所有模型中表现最优。研究推动了基于先进方法(如迁移学习与Transformer)的建筑能耗预测发展。

原文摘要 · Abstract (English)

This study investigates the application of Transfer Learning (TL) on Transformer architectures to enhance building energy consumption forecasting. Transformers are a relatively new deep learning architecture, which has served as the foundation for groundbreaking technologies such as ChatGPT. While TL has been studied in the past, prior studies considered either one data-centric TL strategy or used older deep learning models such as Recurrent Neural Networks or Convolutional Neural Networks. Here, we carry out an extensive empirical study on six different data-centric TL strategies and analyse their performance under varying feature spaces. In addition to the vanilla Transformer architecture, we also experiment with Informer and PatchTST, specifically designed for time series forecasting. We use 16 datasets from the Building Data Genome Project 2 to create building energy consumption forecasting models. Experimental results reveal that while TL is generally beneficial, especially when the target domain has no data, careful selection of the exact TL strategy should be made to gain the maximum benefit. This decision largely depends on the feature space properties such as the recorded weather features. We also note that PatchTST outperforms the other two Transformer variants (vanilla Transformer and Informer). Our findings advance the building energy consumption forecasting using advanced approaches like TL and Transformer architectures.

迁移学习建筑能耗Transformer时间序列

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。