用注意力机制在特征空间做时间序列预测,让结果更准更紧凑。
Attention-Based Feature Online Conformal Prediction for Time Series
- 在神经网络特征空间计算非契合度,聚焦关键信息
- 通过注意力加权历史数据,预测区间最大缩小88%
- 适合需要高精度与鲁棒性的时序预测场景
在线共形预测(OCP)可为任意预训练模型提供覆盖保证,不受时间依赖或分布偏移影响。但传统OCP存在两大局限:仅在输出空间使用简单非契合度分数,且对历史观测一视同仁地估计分位数。本文提出基于注意力的特征空间在线共形预测(AFOCP),通过两项创新解决上述问题:首先,AFOCP在预训练神经网络的特征空间中操作,利用学习到的表示构建更紧凑的预测集,聚焦任务相关特征并抑制无关变化;其次,引入注意力机制,根据历史观测与当前测试点的相关性自适应加权,有效应对非平稳性和分布漂移。理论分析表明,在温和正则条件下,AFOCP保持长期覆盖性,且预测区间明显小于标准OCP。大量合成与真实时间序列实验显示,相比OCP,AFOCP在维持目标覆盖率的同时,预测区间最大缩短88%。
原文摘要 · Abstract (English)
Online conformal prediction (OCP) wraps around any pre-trained predictor to produce prediction sets with coverage guarantees that hold irrespective of temporal dependencies or distribution shifts. However, standard OCP faces two key limitations: it operates in the output space using simple nonconformity (NC) scores, and it treats all historical observations uniformly when estimating quantiles. This paper introduces attention-based feature OCP (AFOCP), which addresses both limitations through two key innovations. First, AFOCP operates in the feature space of pre-trained neural networks, leveraging learned representations to construct more compact prediction sets by concentrating on task-relevant information while suppressing nuisance variation. Second, AFOCP incorporates an attention mechanism that adaptively weights historical observations based on their relevance to the current test point, effectively handling non-stationarity and distribution shifts. We provide theoretical guarantees showing that AFOCP maintains long-term coverage while provably achieving smaller prediction intervals than standard OCP under mild regularity conditions. Extensive experiments on synthetic and real-world time series datasets demonstrate that AFOCP consistently reduces the size of prediction intervals by as much as $88\%$ as compared to OCP, while maintaining target coverage levels, validating the benefits of both feature-space calibration and attention-based adaptive weighting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。