arXiv:2501.09396eess.IVcs.CV2025-01

用事件数据联合传输模糊图像,实现低带宽下的高清复原。

Joint Transmission and Deblurring: A Semantic Communication Approach Using Events

  • 分离共享与特定信息,分途传输模糊图像和事件数据
  • 多阶段训练使重建质量显著优于现有方法
  • 适合运动模糊场景下高效图像传输的科研与工程应用

基于深度学习的联合源信道编码(JSCC)正成为高效图像传输的前沿技术。然而,现有方法多聚焦于清晰图像传输,忽视了由相机抖动或快速运动物体引起的运动模糊这一真实世界挑战。运动模糊会严重降低图像质量,增加传输与重建难度。事件相机通过异步记录像素亮度变化,具有极低延迟,已在运动去模糊任务中展现出巨大潜力。但其产生的海量数据如何高效传输仍是一大难题。本文提出一种新型JSCC框架,用于模糊图像与事件数据的联合传输,旨在有限信道带宽下实现高质量重建。该系统以去模糊任务为导向,利用RGB相机与事件相机从不同模态捕捉同一场景的特点,分别提取并传输共享信息与领域特定信息,避免重复传输。接收端通过去模糊解码器生成清晰图像。此外,引入多阶段训练策略优化模型性能。仿真结果表明,该方法显著优于现有基于JSCC的图像传输方案,有效解决了运动模糊问题。

原文摘要 · Abstract (English)

Deep learning-based joint source-channel coding (JSCC) is emerging as a promising technology for effective image transmission. However, most existing approaches focus on transmitting clear images, overlooking real-world challenges such as motion blur caused by camera shaking or fast-moving objects. Motion blur often degrades image quality, making transmission and reconstruction more challenging. Event cameras, which asynchronously record pixel intensity changes with extremely low latency, have shown great potential for motion deblurring tasks. However, the efficient transmission of the abundant data generated by event cameras remains a significant challenge. In this work, we propose a novel JSCC framework for the joint transmission of blurry images and events, aimed at achieving high-quality reconstructions under limited channel bandwidth. This approach is designed as a deblurring task-oriented JSCC system. Since RGB cameras and event cameras capture the same scene through different modalities, their outputs contain both shared and domain-specific information. To avoid repeatedly transmitting the shared information, we extract and transmit their shared information and domain-specific information, respectively. At the receiver, the received signals are processed by a deblurring decoder to generate clear images. Additionally, we introduce a multi-stage training strategy to train the proposed model. Simulation results demonstrate that our method significantly outperforms existing JSCC-based image transmission schemes, addressing motion blur effectively.

图像传输事件相机去模糊联合编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。