用可学习查表法实现高效遥感图像融合,轻量快速且保真度高。
Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables
- 设计三类可学习查表,分别处理光谱映射、空间细节和通道信息融合。
- 参数少于70万,在2080 Ti上9K×9K图像处理时间不足1毫秒。
- 可在24GB显存下处理15K×15K大图,适合无专用设备的现实场景。
近期基于深度学习的全色锐化方法在性能上超越传统方法,但推理时计算开销大,尤其对大尺寸图像不友好,限制了其在真实场景中的应用,尤其在缺乏GPU/TPU等专用硬件的情况下。为此,我们提出Pan-LUT,一种基于可学习查找表(LUT)的新框架,兼顾性能与效率,实现大尺度遥感图像的全色锐化。为精细控制光谱变换,设计了通道级光谱映射的PAN引导查表(PGLUT);为捕捉细粒度空间细节,引入空间细节查表(SDLUT);为自适应聚合通道信息生成高分辨率多光谱图像,设计自适应输出查表(AOLUT)。模型参数少于70万,使用一张RTX 2080 Ti GPU,9K×9K图像处理时间低于1毫秒,显著快于现有方法。实验表明,该模型能高效处理大尺寸遥感图像,弥补实际应用差距,且在真实世界条件下全分辨率场景中优于当前最优方法。
原文摘要 · Abstract (English)
Recently, deep learning-based pan-sharpening algorithms have achieved notable advancements over traditional methods. However, deep learning-based methods incur substantial computational overhead during inference, especially with large images. This excessive computational demand limits the applicability of these methods in real-world scenarios, particularly in the absence of dedicated computing devices such as GPUs and TPUs. To address these challenges, we propose Pan-LUT, a novel learnable look-up table (LUT) framework for pan-sharpening that strikes a balance between performance and computational efficiency for large remote sensing images. Our method makes it possible to process 15K*15K remote sensing images on a 24GB GPU. To finely control the spectral transformation, we devise the PAN-guided look-up table (PGLUT) for channel-wise spectral mapping. To effectively capture fine-grained spatial details, we introduce the spatial details look-up table (SDLUT). Furthermore, to adaptively aggregate channel information for generating high-resolution multispectral images, we design an adaptive output look-up table (AOLUT). Our model contains fewer than 700K parameters and processes a 9K*9K image in under 1 ms using one RTX 2080 Ti GPU, demonstrating significantly faster performance compared to other methods. Experiments reveal that Pan-LUT efficiently processes large remote sensing images in a lightweight manner, bridging the gap to real-world applications. Furthermore, our model surpasses SOTA methods in full-resolution scenes under real-world conditions, highlighting its effectiveness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。