快速实现跨传感器高保真影像融合,3秒完成一张图像处理。
Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints
- 用预训练模型引导轻量适配网络,联合优化光谱与物理保真度。
- 在WorldView-3上512×512×8图像3秒内完成训练与推理。
- 适合实际部署,兼具强泛化性与高速度,尤其适用于真实场景。
遥感影像融合旨在通过低分辨率多光谱(LRMS)与高分辨率全色(PAN)图像生成高分辨率多光谱(HRMS)图像,同时保留光谱与空间信息。尽管深度学习方法表现优异,但需大量数据和高训练成本,且测试分布偏离时性能下降,泛化能力受限。近期零样本方法虽具强泛化性,但融合质量有限、计算开销大、收敛慢。本文提出FMG-Pan框架,一种快速且通用的模型引导式实例自适应方法,实现跨传感器泛化与高效训练推理。该框架利用预训练模型指导轻量适配网络,联合优化光谱与物理保真约束,并设计新颖物理保真项以增强空间细节保留。在真实数据集上,无论同传感器或跨传感器设置下均达领先性能。在WorldView-3数据集上,对512×512×8图像,于RTX 3090 GPU上3秒内完成训练与推理,显著快于现有零样本方法,具备实用部署潜力。
原文摘要 · Abstract (English)
Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images while preserving both spectral and spatial information. Although deep learning (DL)-based pansharpening methods achieve impressive performance, they require high training cost and large datasets, and often degrade when the test distribution differs from training, limiting generalization. Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong generalization but suffer from limited fusion quality, high computational overhead, and slow convergence. To address these issues, we propose FMG-Pan, a fast and generalizable model-guided instance-wise adaptation framework for real-world pansharpening, achieving both cross-sensor generality and rapid training-inference. The framework leverages a pretrained model to guide a lightweight adaptive network through joint optimization with spectral and physical fidelity constraints. We further design a novel physical fidelity term to enhance spatial detail preservation. Extensive experiments on real-world datasets under both intra- and cross-sensor settings demonstrate state-of-the-art performance. On the WorldView-3 dataset, FMG-Pan completes training and inference for a 512x512x8 image within 3 seconds on an RTX 3090 GPU, significantly faster than existing zero-shot methods, making it suitable for practical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。