用隐式神经表示融合红外与可见光图像,无需训练集且支持超分辨率。
Infrared and Visible Image Fusion Based on Implicit Neural Representations

- 通过神经网络隐式建模多模态图像,以坐标为输入自适应融合特征。
- 在多个指标上优于现有方法,保留热辐射信息并还原纹理细节。
- 无需训练数据,可处理不同分辨率图像并实现超分辨率重建。
红外与可见光图像融合旨在结合两种模态的优势,生成信息丰富且满足视觉或计算需求的图像。本文提出基于隐式神经表示(INR)的图像融合方法INRFuse,通过神经网络参数化连续函数,隐式表示图像的多模态信息,突破传统依赖离散像素或显式特征的局限。将红外与可见光图像的归一化空间坐标作为输入,利用多层感知机自适应融合双模态特征,输出融合图像。通过设计多种损失函数,联合优化融合图像与原始图像的相似性,在有效保留红外图像热辐射信息的同时,维持可见光图像的纹理细节。此外,INR的分辨率无关特性使得该方法可直接融合不同分辨率图像,并通过高密度坐标查询实现超分辨率重建。实验结果表明,INRFuse在主观视觉质量与客观评价指标上均优于现有方法,生成的融合图像结构清晰、细节自然、信息丰富,且无需训练数据。
原文摘要 · Abstract (English)
Infrared and visible light image fusion aims to combine the strengths of both modalities to generate images that are rich in information and fulfill visual or computational requirements. This paper proposes an image fusion method based on Implicit Neural Representations (INR), referred to as INRFuse. This method parameterizes a continuous function through a neural network to implicitly represent the multimodal information of the image, breaking through the traditional reliance on discrete pixels or explicit features. The normalized spatial coordinates of the infrared and visible light images serve as inputs, and multi-layer perceptrons is utilized to adaptively fuse the features of both modalities, resulting in the output of the fused image. By designing multiple loss functions, the method jointly optimizes the similarity between the fused image and the original images, effectively preserving the thermal radiation information of the infrared image while maintaining the texture details of the visible light image. Furthermore, the resolution-independent characteristic of INR allows for the direct fusion of images with varying resolutions and achieves super-resolution reconstruction through high-density coordinate queries. Experimental results indicate that INRFuse outperforms existing methods in both subjective visual quality and objective evaluation metrics, producing fused images with clear structures, natural details, and rich information without the necessity for a training dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。