用视频压缩技术高效降低地球观测数据体积，保持高精度。
Video Compression for Spatiotemporal Earth System Data
- 将多通道时空数据转为视频格式，利用ffmpeg压缩
- 最高压缩250倍，0.1 bpppb下峰值信噪比超40dB
- 适合处理遥感、气象等大规模地球数据的科研人员
大规模地球系统数据，如高分辨率遥感影像和时空气候模型输出，具有与标准视频相似的特性。其固有的空间、时间与光谱冗余可被成熟的视频压缩技术有效利用。本文提出xarrayvideo，一个Python库，通过将多通道时空数据编码为视频来实现压缩。该方法借助ffmpeg中的标准优化编码器，最高可实现250倍压缩比，同时保持高保真度。我们在四个真实数据集上验证了有效性：DynamicEarthNet（极高分辨率Planet影像）、DeepExtremeCubes（高分辨率哨兵2号影像）、ERA5（气象再分析数据）以及SimpleS2（高分辨率多波段哨兵2号影像），在0.1比特每像素每波段（bpppb）下分别达到55.86、40.60、46.58和43.23 dB的峰值信噪比（PSNR），在1 bpppb下则分别为65.91、54.28、62.90和55.04 dB。我们通过HuggingFace以TACO格式重新发布其中两个数据集——DeepExtremeCubes（原2.3 Tb，压缩至270 Gb）和DynamicEarthNet（原525 Gb，压缩至8.5 Gb），质量未下降（PSNR 55.77–56.65 和 60.15）。在下游深度学习任务中（如反射率预测和土地覆盖分类），使用压缩数据无性能损失。xarrayvideo为快速膨胀的地球观测数据提供高效解决方案，使先进压缩技术对地球科学界更易用、实用。代码已开源：https://github.com/IPL-UV/xarrayvideo。
原文摘要 · Abstract (English)
Large-scale Earth system datasets, from high-resolution remote sensing imagery to spatiotemporal climate model outputs, exhibit characteristics analogous to those of standard videos. Their inherent spatial, temporal, and spectral redundancies can thus be readily exploited by established video compression techniques. Here, we present xarrayvideo, a Python library for compressing multichannel spatiotemporal datasets by encoding them as videos. Our approach achieves compression ratios of up to 250x while maintaining high fidelity by leveraging standard, well-optimized video codecs through ffmpeg. We demonstrate the library's effectiveness on four real-world multichannel spatiotemporal datasets: DynamicEarthNet (very high resolution Planet images), DeepExtremeCubes (high resolution Sentinel-2 images), ERA5 (weather reanalysis data), and the SimpleS2 dataset (high resolution multichannel Sentinel-2 images), achieving Peak Signal-to-Noise Ratios (PSNRs) of 55.86, 40.60, 46.58, and 43.23 dB at 0.1 bits per pixel per band (bpppb) and 65.91, 54.28, 62.90, and 55.04 dB at 1 bpppb. We are redistributing two of these datasets, DeepExtremeCubes (2.3 Tb) and DynamicEarthNet (525 Gb), in the machine-learning-ready and cloud-ready TACO format through HuggingFace at significantly reduced sizes (270 Gb and 8.5 Gb, respectively) without compromising quality (PSNR 55.77-56.65 and 60.15). No performance loss is observed when the compressed versions of these datasets are used in their respective deep learning-based downstream tasks (next step reflectance prediction and landcover segmentation). In conclusion, xarrayvideo presents an efficient solution for handling the rapidly growing size of Earth observation datasets, making advanced compression techniques accessible and practical to the Earth science community. The library is available for use at https://github.com/IPL-UV/xarrayvideo
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。