快速适配不同传感器的遥感图像融合方法,仅需1分钟即可达到全量训练效果。
SWIFT: A General Sensitive Weight Identification Framework for Fast Sensor-Transfer Pansharpening
- 通过数据流形采样选出3%关键样本,高效定位对传感器差异敏感的权重。
- 在单张RTX 4090 GPU上将适应时间从数小时缩短至约1分钟。
- 可插件式适配现有模型,适合需要快速部署的遥感图像处理场景。
多光谱图像融合旨在将高分辨率全色(PAN)图像与低分辨率多光谱(LRMS)图像融合,生成高分辨率多光谱(HRMS)图像。尽管基于深度学习的方法表现优异,但在面对未见过的传感器数据时性能显著下降。传统全量重训练或设计复杂架构成本高昂,难以实际应用。为此,本文提出SWIFT(快速敏感权重识别框架),通过无监督的数据流形采样策略,在目标域中仅选取3%最具信息量的样本,避免传统最远点采样的偏差。利用这些样本分析源域预训练模型参数的梯度行为,快速识别并更新对域偏移最敏感的权重子集。作为即插即用框架,SWIFT可适配多种现有融合模型。大量实验表明,其将适应时间从数小时压缩至约1分钟(单张NVIDIA RTX 4090 GPU),且性能显著优于直接迁移基线,在WorldView-2和QuickBird数据集上达到甚至超过全量重训练的效果,建立跨传感器图像融合新基准。
原文摘要 · Abstract (English)
Pansharpening aims to fuse high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images to generate high-resolution multispectral (HRMS) images. Although deep learning-based methods have achieved promising performance, they generally suffer from severe performance degradation when applied to data from unseen sensors. Adapting these models through full-scale retraining or designing more complex architectures is often prohibitively expensive and impractical for real-world deployment. To address this critical challenge, we propose a fast and general-purpose framework for cross-sensor adaptation, SWIFT (Sensitive Weight Identification for Fast Transfer). Specifically, SWIFT employs an unsupervised sampling strategy based on data manifold structures to balance sample selection while mitigating the bias of traditional Farthest Point Sampling, efficiently selecting only 3\% of the most informative samples from the target domain. This subset is then used to probe a source-domain pre-trained model by analyzing the gradient behavior of its parameters, allowing for the quick identification and subsequent update of only the weight subset most sensitive to the domain shift. As a plug-and-play framework, SWIFT can be applied to various existing pansharpening models. Extensive experiments demonstrate that SWIFT reduces the adaptation time from hours to approximately one minute on a single NVIDIA RTX 4090 GPU. The adapted models not only substantially outperform direct-transfer baselines but also achieve performance competitive with, and in some cases superior to, full retraining, establishing a new state-of-the-art on cross-sensor pansharpening tasks for the WorldView-2 and QuickBird datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。