轻量化压缩卫星图像,用扩散模型还原细节,提升传输效率。
COSMIC: Compress Satellite Images Efficiently via Diffusion Compensation
- 轻量编码器降低2.6~5倍计算量,适配卫星嵌入式设备。
- 地面端用扩散模型补偿细节,重建质量优于现有方法。
- 利用坐标时间等多模态信息引导生成,适合遥感图像场景。
随着在轨卫星数量激增及其能力提升,卫星获取的地球观测图像数量已超出星地链路的传输能力。现有学习型图像压缩方案虽通过复杂编码器提取丰富特征实现高压缩率,但难以部署于算力与功耗受限的卫星嵌入式GPU。本文提出COSMIC,一种高效轻量的卫星图像压缩方案:在卫星端设计轻量编码器,将浮点运算量减少2.6~5倍,显著提升压缩比;在地面端,为弥补简化编码器导致的特征提取能力下降,提出基于扩散模型的细节补偿机制。核心洞察是卫星遥感图像本质为多模态数据,可结合位置、时间等传感器信息作为扩散生成的条件输入。大量实验表明,COSMIC在感知质量和失真指标上均超越现有最优基线。
原文摘要 · Abstract (English)
With the rapidly increasing number of satellites in space and their enhanced capabilities, the amount of earth observation images collected by satellites is exceeding the transmission limits of satellite-to-ground links. Although existing learned image compression solutions achieve remarkable performance by using a sophisticated encoder to extract fruitful features as compression and using a decoder to reconstruct, it is still hard to directly deploy those complex encoders on current satellites' embedded GPUs with limited computing capability and power supply to compress images in orbit. In this paper, we propose COSMIC, a simple yet effective learned compression solution to transmit satellite images. We first design a lightweight encoder (i.e. reducing FLOPs by 2.6~5x) on satellite to achieve a high image compression ratio to save satellite-to-ground links. Then, for reconstructions on the ground, to deal with the feature extraction ability degradation due to simplifying encoders, we propose a diffusion-based model to compensate image details when decoding. Our insight is that satellite's earth observation photos are not just images but indeed multi-modal data with a nature of Text-to-Image pairing since they are collected with rich sensor data (e.g. coordinates, timestamp, etc.) that can be used as the condition for diffusion generation. Extensive experiments show that COSMIC outperforms state-of-the-art baselines on both perceptual and distortion metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。