用地图引导扩散模型,实现超低码率遥感图像高质量重建。
Map-Assisted Remote-Sensing Image Compression at Extremely Low Bitrates
- 用预训练扩散模型结合矢量地图提供语义结构指导
- 在极低码率下实现视觉逼真且语义准确的图像还原
- 适合遥感图像压缩、边缘设备存储与窄带传输场景
在边缘设备存储和窄带传输等实际场景中,超低码率遥感(RS)图像压缩始终是一项挑战。尽管生成模型如变分自编码器(VAEs)和生成对抗网络(GANs)已被用于将遥感图像压缩至极低码率流,但这些模型因超低码率下的高度病态性而难以生成视觉上可信的图像。为此,我们提出一种利用具备强大自然图像先验的预训练扩散模型的图像压缩框架,以实现高真实感重建。然而,扩散模型在低码率下因信息严重丢失容易产生小结构和纹理幻觉。因此,我们引入矢量地图作为语义和结构引导,提出一种名为地图辅助生成压缩(MAGC)的新方法。MAGC采用两阶段流程,在极低码率下压缩与解压遥感图像:第一阶段将图像映射为潜在表示,并通过变分自编码器进一步压缩以节省比特数,同时作为后续扩散过程的隐式引导;第二阶段使用条件扩散模型,结合隐式引导与显式语义引导,生成视觉愉悦且语义准确的结果。定量与定性对比表明,该方法在感知质量和语义准确性方面优于标准编解码器及其他学习型方法。数据集与代码将公开于 https://github.com/WHUyyx/MAGC。
原文摘要 · Abstract (English)
Remote-sensing (RS) image compression at extremely low bitrates has always been a challenging task in practical scenarios like edge device storage and narrow bandwidth transmission. Generative models including VAEs and GANs have been explored to compress RS images into extremely low-bitrate streams. However, these generative models struggle to reconstruct visually plausible images due to the highly ill-posed nature of extremely low-bitrate image compression. To this end, we propose an image compression framework that utilizes a pre-trained diffusion model with powerful natural image priors to achieve high-realism reconstructions. However, diffusion models tend to hallucinate small structures and textures due to the significant information loss at limited bitrates. Thus, we introduce vector maps as semantic and structural guidance and propose a novel image compression approach named Map-Assisted Generative Compression (MAGC). MAGC employs a two-stage pipeline to compress and decompress RS images at extremely low bitrates. The first stage maps an image into a latent representation, which is then further compressed in a VAE architecture to save bitrates and serves as implicit guidance in the subsequent diffusion process. The second stage conducts a conditional diffusion model to generate a visually pleasing and semantically accurate result using implicit guidance and explicit semantic guidance. Quantitative and qualitative comparisons show that our method outperforms standard codecs and other learning-based methods in terms of perceptual quality and semantic accuracy. The dataset and code will be publicly available at https://github.com/WHUyyx/MAGC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。