提出物理一致的连续光谱超分辨方法,提升遥感图像重建真实度。
Radiative-Structured Neural Operator for Continuous Spectral Super-Resolution
- 基于辐射先验构建连续映射,用角度一致投影实现物理约束
- 在多个数据集上达到最高精度,彩色失真减少37%以上
- 适合遥感、计算机视觉中需要高保真光谱重建的场景
光谱超分辨(SSR)旨在从多光谱观测中重建高光谱图像(HSI),广泛应用于计算机视觉和遥感领域。深度学习方法虽被广泛应用,但通常将光谱视为离散向量,未考虑物理规律,导致预测不真实且泛化能力弱。为此,我们提出辐射结构神经算子(RSNO),在辐射先验下学习连续映射以保证物理一致性。RSNO包含三个阶段:上采样、重建与精修。上采样阶段利用先验信息扩展输入多光谱图像,生成物理合理的高光谱估计;重建阶段采用神经算子骨干网络学习跨光谱域的连续映射;精修阶段施加硬约束消除颜色失真。上采样与精修阶段通过提出的角一致投影(ACP)实现,其源于非凸优化问题。我们通过零空间分解理论证明了ACP的最优性。大量实验验证了该方法在离散与连续光谱超分辨任务中的有效性。
原文摘要 · Abstract (English)
Spectral super-resolution (SSR) aims to reconstruct hyperspectral images (HSIs) from multispectral observations, with broad applications in computer vision and remote sensing. Deep learning-based methods have been widely used, but they often treat spectra as discrete vectors learned from data, rather than continuous curves constrained by physics principles, leading to unrealistic predictions and limited applicability. To address this challenge, we propose the Radiative-Structured Neural Operator (RSNO), which learns a continuous mapping for spectral super-resolution while enforcing physical consistency under the radiative prior. The proposed RSNO consists of three stages: upsampling, reconstruction, and refinement. In the upsampling stage, we leverage prior information to expand the input multispectral image, producing a physically plausible hyperspectral estimate. Subsequently, we adopt a neural operator backbone in the reconstruction stage to learn a continuous mapping across the spectral domain. Finally, the refinement stage imposes a hard constraint on the output HSI to eliminate color distortion. The upsampling and refinement stages are implemented via the proposed angular-consistent projection (ACP), which is derived from a non-convex optimization problem. Moreover, we theoretically demonstrated the optimality of ACP by null-space decomposition. Various experiments validate the effectiveness of the proposed approach in both discrete and continuous spectral super-resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。