针对科学数据高保真压缩,提出专用于残差的编码方法,显著提升压缩率。
Residual Modeling for High-Fidelity Learned Compression of Scientific Data

- 将学习到的残差视为独立结构,设计专用编码器进行高效压缩。
- 在10^-6至10^-4的块级NRMSE目标下,压缩率比传统方法提升30%-60%。
- 适合对精度要求极高的科学模拟数据压缩,如气候、流体仿真等场景。
科学模拟产生的大规模时空数据需要有损压缩。学习型压缩器可在中等精度目标下实现高压缩比,但其整体重建误差无法保证每一块的精度。现有保障自编码器(GAE)通过保留SVD/PCA式系数逐块修正,但在高保真场景(块级NRMSE 10^-6~10^-4)下,需保留系数过多,修正流主导总码率。本文提出以残差为中心的新视角:学习到的残差结构不同于原始科学场,应采用专门设计的表示进行编码。提出两种残差编码器:LBRC为无训练、确定性流水线,自适应量化残差至目标NRMSE,再用3D Lorenzo差分、之字映射、比特平面编码与熵编码无损压缩整数残差;NGLR在相同流水线中加入因果神经预测器,输出归一化偏置修正整数舍入的Lorenzo预测,降低剩余残差熵,预测权重序列化并计入码流。在E3SM、JHTDB、ERA5数据集上,块级NRMSE目标为10^-6至10^-4时,LBRC相比GAE压缩率提升30%-60%,与SZ相当;NGLR进一步提升10%-40%,在高保真条件下优于SZ。结果表明,针对学习压缩残差设计的表示可避免全局残差修正导致的码率膨胀,保留学习压缩优势。
原文摘要 · Abstract (English)
Lossy compression is essential for massive spatiotemporal data from scientific simulations. Learned compressors can achieve high compression ratios at moderate accuracy targets, but their aggregate reconstruction losses do not guarantee accuracy for each block. Existing Guaranteed Autoencoder (GAE) methods add a per-block residual correction by retaining SVD/PCA-style coefficients until the target is met. This works at moderate tolerances, but in the high-fidelity regime with block-level NRMSE from 10^-6 to 10^-4, the number of retained coefficients grows quickly and the correction stream dominates the total rate. We propose a residual-centric view: the learned residual is structurally different from the original scientific field and should be coded with a representation designed for that residual. We introduce two residual coders. LBRC is a deterministic, training-free pipeline that adaptively quantizes the learned residual to the target NRMSE and losslessly encodes the resulting integer residual using 3D Lorenzo differencing, zigzag mapping, bit-plane coding, and entropy coding. NGLR adds a causal neural predictor that outputs a normalized bias for an integer-rounded Lorenzo prediction in the same deterministic integer pipeline, reducing the entropy of the remaining residual code while preserving deterministic decoding. The predictor weights are serialized and counted in the bitstream. Across E3SM, JHTDB, and ERA5 at block-level NRMSE targets from 10^-6 to 10^-4, LBRC improves compression ratio over GAE by 30-60% and is broadly competitive with SZ. NGLR adds a further 10-40% over LBRC and outperforms SZ in the evaluated high-fidelity regime. These results show that residual representations tailored to learned-compressor residuals can preserve the advantage of learned compression when global residual correction becomes rate-dominant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。