用像素混洗生成训练数据,让模型学会融合多焦点图像。
Inter-Image Pixel Shuffling for Multi-focus Image Fusion
- 通过混洗清晰图与模糊图的像素,构造无真实多焦点图的训练数据。
- 在多个数据集上超越现有方法,尤其在未见场景下表现更优。
- 适合做图像融合、计算机视觉任务的研究者和工程师参考。
多焦点图像融合旨在将多张部分聚焦的图像合成一张全清晰图像。尽管深度学习在此任务中展现出潜力,但其效果常受限于合适训练数据的稀缺。本文提出跨图像像素混洗(IPS),一种无需真实多焦点图像即可训练神经网络的新方法。IPS将任务重构为像素级分类问题:在每个空间位置上,判断哪一像素是清晰的。具体地,将原始图像的像素视为清晰,其低通滤波版本的像素视为模糊,并随机混洗二者在同一位置的像素,生成保留空间结构但混合聚焦信息的训练样本。模型学习从每组对齐像素中选择清晰像素,从而聚合输入图像中的锐利内容以重建全清晰图像。为提升融合质量,IPS采用跨图像融合网络,结合卷积神经网络的局部表征能力与状态空间模型的长程建模能力,有效利用空间细节与上下文信息。实验表明,即使不使用真实多焦点图像训练,IPS也显著优于现有方法。
原文摘要 · Abstract (English)
Multi-focus image fusion aims to combine multiple partially focused images into a single all-in-focus image. Although deep learning has shown promise in this task, its effectiveness is often limited by the scarcity of suitable training data. This paper introduces Inter-image Pixel Shuffling (IPS), a novel method that allows neural networks to learn multi-focus image fusion without requiring actual multi-focus images. IPS reformulates the task as a pixel-wise classification problem, where the goal is to identify the focused pixel from a pixel group at each spatial position. In this method, pixels from a clear optical image are treated as focused, while pixels from a low-pass filtered version of the same image are considered defocused. By randomly shuffling the focused and defocused pixels at identical spatial positions in the original and filtered images, IPS generates training data that preserves spatial structure while mixing focus-defocus information. The model is trained to select the focused pixel from each spatially aligned pixel group, thus learning to reconstruct an all-in-focus image by aggregating sharp content from the input. To further enhance fusion quality, IPS adopts a cross-image fusion network that integrates the localized representation power of convolutional neural networks with the long-range modeling capabilities of state space models. This design effectively leverages both spatial detail and contextual information to produce high-quality fused results. Experimental results indicate that IPS significantly outperforms existing multi-focus image fusion methods, even without training on multi-focus images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。