用潜在扩散模型提升图像传输质量,降低计算开销。
CASC: Condition-Aware Semantic Communication with Latent Diffusion Models
- 接收端用噪声隐空间作为条件,重建源图像。
- 相比原有方法,感知质量显著提升,推理速度加快51.7%。
- 动态调整网络权重,适合对图像质量要求高的场景。
基于扩散模型的语义通信在图像传输中展现出显著优势,但存在生成随机性导致重建失真及计算成本高的问题。为此,我们提出CASC框架,采用潜在扩散模型(LDM)构建解码器。接收端利用接收到的噪声隐空间编码作为条件信号,实现隐空间重构,使解码器更准确地恢复原始图像。通过在隐空间操作,该方法降低了传统扩散模型的计算复杂度。此外,引入条件感知神经网络(CAN),根据条件信号动态调整LDM隐藏层权重,实现更精细的生成控制,显著提升重建图像的感知质量。实验表明,CASC在感知质量与视觉效果上均优于DeepJSCC;同时,相比现有基于扩散模型的通信系统,推理时间减少51.7%,且保持相近的感知性能。消融实验证实了CAN模块对图像重建质量的有效提升。
原文摘要 · Abstract (English)
Diffusion-based semantic communication methods have shown significant advantages in image transmission by harnessing the generative power of diffusion models. However, they still face challenges, including generation randomness that leads to distorted reconstructions and high computational costs. To address these issues, we propose CASC, a condition-aware semantic communication framework that incorporates a latent diffusion model (LDM)-based denoiser. The LDM denoiser at the receiver utilizes the received noisy latent codes as the conditioning signal to reconstruct the latent codes, enabling the decoder to accurately recover the source image. By operating in the latent space, the LDM reduces computational complexity compared to traditional diffusion models (DMs). Additionally, we introduce a condition-aware neural network (CAN) that dynamically adjusts the weights in the hidden layers of the LDM based on the conditioning signal. This enables finer control over the generation process, significantly improving the perceptual quality of the reconstructed images. Experimental results show that CASC significantly outperforms DeepJSCC in both perceptual quality and visual effect. Moreover, CASC reduces inference time by 51.7% compared to existing DM-based semantic communication systems, while maintaining comparable perceptual performance. The ablation studies also validate the effectiveness of the CAN module in improving the image reconstruction quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。