用小波编码与频域-空间双域门控,提升视频预测的细节和长期动态建模能力。
WaveSFNet: A Wavelet-Based Codec and Spatial--Frequency Dual-Domain Gating Network for Spatiotemporal Prediction
- 结合小波编码与双域门控,保留高频纹理并建模长程动态。
- 在Moving MNIST等数据集上实现高精度多步预测,计算量低。
- 适合需要高分辨率、长序列视频预测的场景,如气象与交通模拟。
时空预测旨在无监督条件下从历史观测中预测未来帧,广泛应用于各类场景。核心挑战在于建模长程动态的同时保持高频细节以实现清晰的多步预测。现有高效非循环框架通常依赖下采样卷积或池化,易丢失纹理与边界信息;而纯空间操作难以兼顾局部交互与全局传播。为此,我们提出WaveSFNet,一个融合小波编码器与时空双域门控译码器的高效框架。小波编码器在下采样与重构过程中保留高频子带特征。译码器先注入相邻帧差分以增强动态信息,再在大核空间局部建模与频域全局调制间进行门控融合,并通过门控通道交互实现跨通道特征交换。大量实验表明,WaveSFNet在Moving MNIST、TaxiBJ和WeatherBench数据集上均达到领先预测精度,同时保持较低计算复杂度。代码已开源:https://github.com/fhjdqaq/WaveSFNet。
原文摘要 · Abstract (English)
Spatiotemporal predictive learning aims to forecast future frames from historical observations in an unsupervised manner, and is critical to a wide range of applications. The key challenge is to model long-range dynamics while preserving high-frequency details for sharp multi-step predictions. Existing efficient recurrent-free frameworks typically rely on strided convolutions or pooling for sampling, which tends to discard textures and boundaries, while purely spatial operators often struggle to balance local interactions with global propagation. To address these issues, we propose WaveSFNet, an efficient framework that unifies a wavelet-based codec with a spatial--frequency dual-domain gated spatiotemporal translator. The wavelet-based codec preserves high-frequency subband cues during downsampling and reconstruction. Meanwhile, the translator first injects adjacent-frame differences to explicitly enhance dynamic information, and then performs dual-domain gated fusion between large-kernel spatial local modeling and frequency-domain global modulation, together with gated channel interaction for cross-channel feature exchange. Extensive experiments demonstrate that WaveSFNet achieves competitive prediction accuracy on Moving MNIST, TaxiBJ, and WeatherBench, while maintaining low computational complexity. Our code is available at https://github.com/fhjdqaq/WaveSFNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。