用贝叶斯注意力提升电网负荷预测的不确定性估计能力
Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids
- 在PatchTST架构中融合三种贝叶斯机制，量化模型参数与注意力不确定性
- 24小时预测下CRPS达0.0289，90%置信区间覆盖率达90.4%，优于现有方法
- 在极端天气下仍保持稳定校准，适合电力系统风险决策应用
现代电网运行需要具有良好校准的不确定性估计的概率负荷预测。然而，现有深度学习模型会产生过于自信的点预测，在极端天气分布偏移下表现灾难性失败。本文提出贝叶斯Transformer（BT）框架，将三种互补的不确定性机制集成到PatchTST主干网络：蒙特卡洛丢弃用于认知参数不确定性，带对数均匀权重先验的变分前馈层，以及可学习高斯噪声扰动预软最大值逻辑的随机注意力，据我们所知，这是首次将贝叶斯注意力应用于概率负荷预测。采用七级多分位数针球损失预测头和训练后等距回归校准，生成精确且接近名义覆盖率的预测区间。在五个电网数据集（PJM、ERCOT、ENTSO-E德国、法国和英国）上，结合NOAA协变量，在24、48和168小时预测时长下进行评估，表现达到最先进水平。在主要基准（PJM，H=24h）上，BT的CRPS为0.0289，相比深度集成提高7.4%，相比确定性LSTM提升29.9%，90%名义水平下预测区间覆盖率为90.4%，且预测区间宽度（4,960 MW）在所有概率基线中最小。在热浪和寒潮事件中，BT分别保持89.6%和90.1%的覆盖率，而确定性LSTM仅为64.7%和67.2%，证实贝叶斯认知不确定性能自然扩大分布外输入的区间。所有时长下校准性能稳定（89.8%-90.4% PICP），消融实验表明各组件均贡献独特价值。校准输出可直接支持基于风险的备用容量配置、随机机组组合和需求响应激活。
原文摘要 · Abstract (English)
The reliable operation of modern power grids requires probabilistic load forecasts with well-calibrated uncertainty estimates. However, existing deep learning models produce overconfident point predictions that fail catastrophically under extreme weather distributional shifts. This study proposes a Bayesian Transformer (BT) framework that integrates three complementary uncertainty mechanisms into a PatchTST backbone: Monte Carlo Dropout for epistemic parameter uncertainty, variational feed-forward layers with log-uniform weight priors, and stochastic attention with learnable Gaussian noise perturbations on pre-softmax logits, representing, to the best of our knowledge, the first application of Bayesian attention to probabilistic load forecasting. A seven-level multi-quantile pinball-loss prediction head and post-training isotonic regression calibration produce sharp, near-nominally covered prediction intervals. Evaluation of five grid datasets (PJM, ERCOT, ENTSO-E Germany, France, and Great Britain) augmented with NOAA covariates across 24, 48, and 168-hour horizons demonstrates state-of-the-art performance. On the primary benchmark (PJM, H=24h), BT achieves a CRPS of 0.0289, improving 7.4% over Deep Ensembles and 29.9% over the deterministic LSTM, with 90.4% PICP at the 90% nominal level and the narrowest prediction intervals (4,960 MW) among all probabilistic baselines. During heat-wave and cold snap events, BT maintained 89.6% and 90.1% PICP respectively, versus 64.7% and 67.2% for the deterministic LSTM, confirming that Bayesian epistemic uncertainty naturally widens intervals for out-of-distribution inputs. Calibration remained stable across all horizons (89.8-90.4% PICP), while ablation confirmed that each component contributed a distinct value. The calibrated outputs directly support risk-based reserve sizing, stochastic unit commitment, and demand response activation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。