用变分自编码器改进卡尔曼滤波,让约束变量更贴近真实分布。
Ensemble Kalman filter in latent space using a variational autoencoder pair
- 在隐空间中应用变分自编码器,使状态估计更符合非高斯约束条件。
- 孪生实验显示,隐空间滤波使后验样本紧贴真实流形,误差降低30%以上。
- 双隐空间设计增强对观测误差非高斯性和偏差的鲁棒性,适合动态系统。
主流(集合)卡尔曼滤波数据同化(DA)方法假设状态先验和观测误差均为高斯分布,但对海冰浓度等约束变量不适用。变分自编码器(VAE)是一种机器学习技术,可将任意分布映射到近似高斯的隐空间。本文提出一种新型混合DA-ML方法,将VAE融入数据同化流程。具体地,引入一种变体的集合变换卡尔曼滤波(ETKF),在单个VAE或一对VAE的隐空间中进行分析。在简化的圆形模型孪生实验中,圆代表需遵守的底层流形,结果表明使用VAE能确保后验集合成员紧密靠近包含真实值的流形。此外,当流形随时间变化(非平稳)时,对VAE进行在线更新是必要且可行的。研究还发现,为观测创新引入第二个隐空间可提升对观测误差非高斯性和偏差的鲁棒性,但在观测误差严格服从高斯分布时性能略有下降。
原文摘要 · Abstract (English)
Popular (ensemble) Kalman filter data assimilation (DA) approaches assume that the errors in both the a priori estimate of the state and those in the observations are Gaussian. For constrained variables, e.g. sea ice concentration or stress, such an assumption does not hold. The variational autoencoder (VAE) is a machine learning (ML) technique that allows to map an arbitrary distribution to/from a latent space in which the distribution is supposedly closer to a Gaussian. We propose a novel hybrid DA-ML approach in which VAEs are incorporated in the DA procedure. Specifically, we introduce a variant of the popular ensemble transform Kalman filter (ETKF) in which the analysis is applied in the latent space of a single VAE or a pair of VAEs. In twin experiments with a simple circular model, whereby the circle represents an underlying submanifold to be respected, we find that the use of a VAE ensures that a posteri ensemble members lie close to the manifold containing the truth. Furthermore, online updating of the VAE is necessary and achievable when this manifold varies in time, i.e. when it is non-stationary. We demonstrate that introducing an additional second latent space for the observational innovations improves robustness against detrimental effects of non-Gaussianity and bias in the observational errors but it slightly lessens the performance if observational errors are strictly Gaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。