arXiv:2503.06623cs.CV2025-03被引 5

将气象数据从像素空间转为隐空间,大幅降低存储与计算成本。

Transforming Weather Data from Pixel to Latent Space

  • 构建天气隐自编码器WLA,实现像素到隐空间的高效转换
  • 压缩后数据仅0.43TB(原244.34TB),保留多压强变量统一表示
  • 隐空间建模显著提升任务精度与清晰度,适合多场景应用

气候变化与极端天气事件加剧推动了深度学习在气象研究中的应用。然而,现有方法多依赖像素空间的气象数据,面临输出平滑、仅适用于单一压强变量子集(PVS)、存储与计算开销高等问题。为此,本文提出新型天气隐自编码器(WLA),将气象数据从像素空间映射至隐空间,实现高效任务建模。通过解耦重建与下游任务,显著提升模型结果的准确性和清晰度。引入压强变量统一模块,将多个PVS融合为统一表示,增强模型在多种气象场景下的适应性。此外,下游任务可在低存储的隐空间中完成,极大降低数据存储与计算成本。实验表明,该方法具备优越的压缩与重建性能,成功构建了包含多个PVS统一表示的ERA5-latent数据集。压缩后的完整PVS数据由原始244.34 TB降至0.43 TB。下游任务验证了隐空间模型可跨多PVS应用,且性能优于像素空间模型。代码、数据集及预训练模型已开源。

原文摘要 · Abstract (English)

The increasing impact of climate change and extreme weather events has spurred growing interest in deep learning for weather research. However, existing studies often rely on weather data in pixel space, which presents several challenges such as smooth outputs in model outputs, limited applicability to a single pressure-variable subset (PVS), and high data storage and computational costs. To address these challenges, we propose a novel Weather Latent Autoencoder (WLA) that transforms weather data from pixel space to latent space, enabling efficient weather task modeling. By decoupling weather reconstruction from downstream tasks, WLA improves the accuracy and sharpness of weather task model results. The incorporated Pressure-Variable Unified Module transforms multiple PVS into a unified representation, enhancing the adaptability of the model in multiple weather scenarios. Furthermore, weather tasks can be performed in a low-storage latent space of WLA rather than a high-storage pixel space, thus significantly reducing data storage and computational costs. Through extensive experimentation, we demonstrate its superior compression and reconstruction performance, enabling the creation of the ERA5-latent dataset with unified representations of multiple PVS from ERA5 data. The compressed full PVS in the ERA5-latent dataset reduces the original 244.34 TB of data to 0.43 TB. The downstream task further demonstrates that task models can apply to multiple PVS with low data costs in latent space and achieve superior performance compared to models in pixel space. Code, ERA5-latent data, and pre-trained models are available at https://anonymous.4open.science/r/Weather-Latent-Autoencoder-8467.

气象建模隐空间数据压缩自编码器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。