提出动态遗忘机制,让时间序列预测模型更关注近期信息。
Learning to Forget: Bayesian Time Series Forecasting using Recurrent Sparse Spectrum Signature Gaussian Processes
- 用衰减的随机傅里叶特征重构签名,实现可学习的遗忘
- 处理10^4步序列仅需0.01秒,内存低于1GB
- 适合需要实时更新、关注近期趋势的预测场景
签名核是任意长度时间序列间的核函数,具有坚实的随机分析理论基础,已在机器学习中作为高斯过程的协方差函数使用。其优势在于提供时间序列的结构化全局描述,但当局部信息重要或需遗忘旧数据时,这一特性反而成为瓶颈。现有方法多依赖人工切片等非系统手段。为此,本文提出一种基于数据驱动的原理性遗忘机制,引入随机傅里叶衰减签名特征(RFDSF),结合高斯过程构建贝叶斯时间序列预测模型,采用变分推断实现端到端训练。该算法在单次递归遍历中完成时间序列处理与未来多步预测分布生成,可高效处理长达10^4步的数据,在约0.01秒内完成,且内存占用低于1GB GPU显存。实验表明,其性能优于其他基于GP的方法,并达到当前主流概率时间序列预测算法水平。
原文摘要 · Abstract (English)
The signature kernel is a kernel between time series of arbitrary length and comes with strong theoretical guarantees from stochastic analysis. It has found applications in machine learning such as covariance functions for Gaussian processes. A strength of the underlying signature features is that they provide a structured global description of a time series. However, this property can quickly become a curse when local information is essential and forgetting is required; so far this has only been addressed with ad-hoc methods such as slicing the time series into subsegments. To overcome this, we propose a principled, data-driven approach by introducing a novel forgetting mechanism for signatures. This allows the model to dynamically adapt its context length to focus on more recent information. To achieve this, we revisit the recently introduced Random Fourier Signature Features, and develop Random Fourier Decayed Signature Features (RFDSF) with Gaussian processes (GPs). This results in a Bayesian time series forecasting algorithm with variational inference, that offers a scalable probabilistic algorithm that processes and transforms a time series into a joint predictive distribution over time steps in one pass using recurrence. For example, processing a sequence of length $10^4$ steps in $\approx 10^{-2}$ seconds and in $< 1\text{GB}$ of GPU memory. We demonstrate that it outperforms other GP-based alternatives and competes with state-of-the-art probabilistic time series forecasting algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。