用隐空间统一表示高维状态与稀疏观测,提升复杂系统预测精度。
Latent-EnSF: A Latent Ensemble Score Filter for High-Dimensional Data Assimilation with Sparse Observation Data
- 通过耦合变分自编码器实现状态与观测的统一隐空间编码
- 在浅水波与中短期天气预报中实现更高精度与更快收敛
- 适合处理高维、稀疏观测的非线性贝叶斯滤波问题
精确建模与预测复杂物理系统常依赖数据同化技术来修正模型模拟中的误差。传统方法如集合卡尔曼滤波(EnKF)及其变体,以及近期发展的集合评分滤波(EnSF),在面对高维且非线性、观测稀疏的贝叶斯滤波问题时面临显著挑战,而这类问题在真实应用中普遍存在。本文提出一种新型数据同化方法——隐空间集成评分滤波(Latent-EnSF),通过高效且一致的隐空间表示,联合处理状态高维性与观测高稀疏性的难题。我们引入一个双编码器耦合的变分自编码器(VAE),以保证状态与观测在隐空间中的分布匹配与重建一致性。在浅水波传播与中短期天气预报两个复杂模型任务中,相比多种方法,Latent-EnSF展现出更高的准确性、更快的收敛速度和更高的计算效率,适用于时空双重稀疏观测场景。
原文摘要 · Abstract (English)
Accurate modeling and prediction of complex physical systems often rely on data assimilation techniques to correct errors inherent in model simulations. Traditional methods like the Ensemble Kalman Filter (EnKF) and its variants as well as the recently developed Ensemble Score Filters (EnSF) face significant challenges when dealing with high-dimensional and nonlinear Bayesian filtering problems with sparse observations, which are ubiquitous in real-world applications. In this paper, we propose a novel data assimilation method, Latent-EnSF, which leverages EnSF with efficient and consistent latent representations of the full states and sparse observations to address the joint challenges of high dimensionlity in states and high sparsity in observations for nonlinear Bayesian filtering. We introduce a coupled Variational Autoencoder (VAE) with two encoders to encode the full states and sparse observations in a consistent way guaranteed by a latent distribution matching and regularization as well as a consistent state reconstruction. With comparison to several methods, we demonstrate the higher accuracy, faster convergence, and higher efficiency of Latent-EnSF for two challenging applications with complex models in shallow water wave propagation and medium-range weather forecasting, for highly sparse observations in both space and time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。