无需真实标签，利用透视几何实现多光谱去马赛克
Perspective-Equivariant Fine-tuning for Multispectral Demosaicing without Ground Truth
- 基于相机成像的投影几何，挖掘更丰富的群结构信息
- 在手术和车载数据集上逼近监督方法性能，还原血管等细节
- 适配预训练模型，直接处理商用传感器原始数据
多光谱去马赛克对从快照马赛克测量中重建全分辨率光谱图像至关重要，广泛应用于神经外科和自动驾驶等实时成像场景。传统方法模糊，而监督学习依赖昂贵的真实标签（GT），需通过缓慢的线扫描系统获取。本文提出透视等变微调框架（PEFD），仅利用马赛克测量即可学习多光谱去马赛克。PEFD a) 利用相机成像系统的投影几何，比以往方法挖掘更丰富的群结构以恢复更多零空间信息；b) 通过适配为1-3通道成像设计的预训练基础模型，无需真实标签高效学习。在手术和汽车数据集上，PEFD能恢复如血管等精细结构并保持光谱保真度，显著优于近期方法，接近监督性能。此外，其在商用多光谱传感器的原始未处理数据上也表现良好。代码见：https://github.com/Andrewwango/pefd。
原文摘要 · Abstract (English)
Multispectral demosaicing is crucial to reconstruct full-resolution spectral images from snapshot mosaiced measurements, enabling real-time imaging from neurosurgery to autonomous driving. Classical methods are blurry, while supervised learning requires costly ground truth (GT) obtained from slow line-scanning systems. We propose Perspective-Equivariant Fine-tuning for Demosaicing (PEFD), a framework that learns multispectral demosaicing from mosaiced measurements alone. PEFD a) exploits the projective geometry of camera-based imaging systems to leverage a richer group structure than previous demosaicing methods to recover more null-space information, and b) learns efficiently without GT by adapting pretrained foundation models designed for 1-3 channel imaging. On surgical and automotive datasets, PEFD recovers fine details such as blood vessels and preserves spectral fidelity, substantially outperforming recent approaches, nearing supervised performance. Furthermore, the performance of PEFD is demonstrated on raw, unprocessed data from a commercial multispectral sensor. Code is at https://github.com/Andrewwango/pefd.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。