用Mamba替代扩散模型中的去噪模块,提升时间序列填补效率与精度
SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputation
- 以Mamba作为扩散模型的去噪主干,实现低复杂度序列建模
- 在多个真实数据集上达到当前最优填补效果,最高提升12.3%
- 适合需要高效处理长序列依赖的时间序列缺失值修复任务
概率性时间序列填补因具备不确定性估计能力而广泛应用。去噪扩散概率模型(DDPMs)凭借对复杂分布的建模能力,在该任务中表现优异。然而现有基于DDPM的方法面临两大挑战:1)去噪部分的主干模块难以实现低时间复杂度的序列建模;2)去噪架构无法有效捕捉时间序列中的依赖关系。针对第一点,本文探索状态空间模型Mamba作为DDPM去噪模块的潜力;针对第二点,设计了若干面向时间序列建模的SSM模块。实验表明,所提方法在多个真实世界数据集上取得领先性能。代码与数据集已开源。
原文摘要 · Abstract (English)
Probabilistic time series imputation has been widely applied in real-world scenarios due to its ability for uncertainty estimation and denoising diffusion probabilistic models~(DDPMs) have achieved great success in probabilistic time series imputation tasks with its power to model complex distributions. However, current DDPM-based probabilistic time series imputation methodologies are confronted with two types of challenges: 1)\textit{The backbone modules of the denoising parts are not capable of achieving sequence modeling with low time complexity.} 2)~\textit{The architecture of denoising modules can not handle the dependencies in the time series data effectively.} To address the first challenge, we explore the potential of state space model, namely Mamba, as the backbone denoising module for DDPMs. To tackle the second challenge, we carefully devise several SSM-based blocks for time series data modeling. Experimental results demonstrate that our approach can achieve state-of-the-art time series imputation results on multiple real-world datasets. Our datasets and code are available at \href{https://github.com/decisionintelligence/SSD-TS/}{https://github.com/decisionintelligence/SSD-TS/}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。