用点云辅助图像压缩，提升纹理与结构保留效果
Point Cloud-Assisted Neural Image Compression
- 融合点云与图像的统一表示，实现多模态信息协同
- 提出MMFFT模块，有效提取关键特征并去除冗余信息
- 首个利用点云提升图像压缩性能的方法，适合自动驾驶场景
高效图像压缩至关重要。在多个传感器采集多模态数据的场景中，现有仅基于图像的编码器未能充分利用其他模态的辅助信息，导致压缩效率不理想。本文通过引入广泛应用于自动驾驶领域的点云数据，提升图像压缩性能。首先统一两种模态的数据表示以利于处理；随后提出点云辅助神经图像编码器（PCA-NIC），利用高维点云信息增强图像纹理与结构的保真度。进一步设计多模态特征融合变换模块（MMFFT），捕捉更具代表性的图像特征，消除通道与模态间无关冗余信息。本工作首次实现以点云提升图像压缩性能，并达到当前最优水平。
原文摘要 · Abstract (English)
High-efficient image compression is a critical requirement. In several scenarios where multiple modalities of data are captured by different sensors, the auxiliary information from other modalities are not fully leveraged by existing image-only codecs, leading to suboptimal compression efficiency. In this paper, we increase image compression performance with the assistance of point cloud, which is widely adopted in the area of autonomous driving. We first unify the data representation for both modalities to facilitate data processing. Then, we propose the point cloud-assisted neural image codec (PCA-NIC) to enhance the preservation of image texture and structure by utilizing the high-dimensional point cloud information. We further introduce a multi-modal feature fusion transform module (MMFFT) to capture more representative image features, remove redundant information between channels and modalities that are not relevant to the image content. Our work is the first to improve image compression performance using point cloud and achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。