arXiv:2604.08903cs.CV2026-04

快速实现跨传感器高保真影像融合,3秒完成一张图像处理。

Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

  • 用预训练模型引导轻量适配网络,联合优化光谱与物理保真度。
  • 在WorldView-3上512×512×8图像3秒内完成训练与推理。
  • 适合实际部署,兼具强泛化性与高速度,尤其适用于真实场景。

遥感影像融合旨在通过低分辨率多光谱(LRMS)与高分辨率全色(PAN)图像生成高分辨率多光谱(HRMS)图像,同时保留光谱与空间信息。尽管深度学习方法表现优异,但需大量数据和高训练成本,且测试分布偏离时性能下降,泛化能力受限。近期零样本方法虽具强泛化性,但融合质量有限、计算开销大、收敛慢。本文提出FMG-Pan框架,一种快速且通用的模型引导式实例自适应方法,实现跨传感器泛化与高效训练推理。该框架利用预训练模型指导轻量适配网络,联合优化光谱与物理保真约束,并设计新颖物理保真项以增强空间细节保留。在真实数据集上,无论同传感器或跨传感器设置下均达领先性能。在WorldView-3数据集上,对512×512×8图像,于RTX 3090 GPU上3秒内完成训练与推理,显著快于现有零样本方法,具备实用部署潜力。

原文摘要 · Abstract (English)

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images while preserving both spectral and spatial information. Although deep learning (DL)-based pansharpening methods achieve impressive performance, they require high training cost and large datasets, and often degrade when the test distribution differs from training, limiting generalization. Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong generalization but suffer from limited fusion quality, high computational overhead, and slow convergence. To address these issues, we propose FMG-Pan, a fast and generalizable model-guided instance-wise adaptation framework for real-world pansharpening, achieving both cross-sensor generality and rapid training-inference. The framework leverages a pretrained model to guide a lightweight adaptive network through joint optimization with spectral and physical fidelity constraints. We further design a novel physical fidelity term to enhance spatial detail preservation. Extensive experiments on real-world datasets under both intra- and cross-sensor settings demonstrate state-of-the-art performance. On the WorldView-3 dataset, FMG-Pan completes training and inference for a 512x512x8 image within 3 seconds on an RTX 3090 GPU, significantly faster than existing zero-shot methods, making it suitable for practical deployment.

影像融合零样本实时处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。