用3D块结构扩散模型实现科学数据压缩,保证最大误差上限。
Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compression
- 3D块压缩+2D去噪网络,利用时空相关性提升压缩效率
- 训练后重建无噪声,结果确定且误差可控
- 适合对误差有严格要求的气候与化学模拟数据
本文提出一种新的有损科学数据压缩范式——带张量修正的保误差条件扩散(GCDTC)。该框架基于条件扩散生成模型,包含条件扩散模型、张量修正和误差保障三部分。扩散模型采用3D条件输入与2D去噪U-Net结合的方式,通过3D块压缩模块捕捉结构化科学数据中的时空相关性。反向扩散过程以压缩模块生成的内容隐变量切片为条件,对2D空间数据进行去噪。训练完成后,去噪解码器在零噪声和内容隐变量下重建数据,实现完全确定性输出。随后通过张量修正与误差保障步骤控制并确保最大误差失真,满足科学数据压缩中不可回避的误差约束。在气候与化学燃烧模拟生成的两个数据集上的实验表明,本方法优于标准卷积自编码器,并在压缩质量上达到现有科学数据压缩算法的竞争力水平。
原文摘要 · Abstract (English)
This paper proposes a new compression paradigm -- Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) -- for lossy scientific data compression. The framework is based on recent conditional diffusion (CD) generative models, and it consists of a conditional diffusion model, tensor correction, and error guarantee. Our diffusion model is a mixture of 3D conditioning and 2D denoising U-Net. The approach leverages a 3D block-based compressing module to address spatiotemporal correlations in structured scientific data. Then, the reverse diffusion process for 2D spatial data is conditioned on the ``slices'' of content latent variables produced by the compressing module. After training, the denoising decoder reconstructs the data with zero noise and content latent variables, and thus it is entirely deterministic. The reconstructed outputs of the CD model are further post-processed by our tensor correction and error guarantee steps to control and ensure a maximum error distortion, which is an inevitable requirement in lossy scientific data compression. Our experiments involving two datasets generated by climate and chemical combustion simulations show that our framework outperforms standard convolutional autoencoders and yields competitive compression quality with an existing scientific data compression algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。