用语义匹配构建伪配对,实现轻量级无配对手机影像处理。
Lightweight Unpaired Smartphone ISP Transfer with Semantic Pseudo-Pairing

- 通过语义嵌入与最优传输构建图像与局部块的伪配对。
- 仅7000参数轻量网络在测试集达22.569 PSNR、0.675 SSIM。
- 适合资源受限场景下的高效无配对图像渲染应用。
由于原始图像(RAW)与目标RGB图像之间缺乏场景和色彩对齐,无配对手机影像信号处理(ISP)是一个挑战性问题。现有方法或依赖成对数据,或严重依赖对抗训练,在无配对设置下易不稳定。本文针对NTIRE 2026无配对数据学习手机ISP挑战,提出一种简单有效的方法:先从训练块重建大图以恢复全局上下文;再使用DINOv2提取语义嵌入,结合融合格罗莫夫-沃瑟斯坦(FGW)最优传输,在图像与块层级建立RAW与RGB间的伪配对。该语义匹配缓解了数据无配对性,构建伪输入-目标对。基于此,训练一个仅含7000参数的轻量CNN进行色彩渲染。网络结构紧凑,专注色彩变换而非结构修改,有助于减少伪影并提升训练稳定性。挑战提交结果在隐藏测试集上取得22.569 PSNR、0.675 SSIM和8.067 ΔE,显著优于基线,且在所有参赛作品中位列第三的SSIM与ΔE表现。代码已开源于github.com/nuniniyujin/Unpaired-ISP。
原文摘要 · Abstract (English)
Unpaired smartphone ISP is a challenging problem due to the lack of scene and color alignment between RAW and target RGB images. Many existing methods either require paired data or rely heavily on adversarial training, which can become unstable in the unpaired setting. In this work, we present a simple and effective approach developed for the NTIRE 2026 Learned Smartphone ISP Challenge with Unpaired Data. Our method first reconstructs larger images from training patches to recover global context. Then, we extract semantic embeddings with DINOv2, and use fused Gromov-Wasserstein (FGW) optimal transport to build pseudo pairs between RAW and RGB images at both image and patch levels. This semantic matching allows us to partially alleviate the unpairedness of the data and build these pseudo input-target pairs. Based on these pseudo pairs, we train a lightweight CNN with only 7K parameters for color rendering. The network is designed to be compact and focus on color transformation rather than structural change, which helps reduce artifacts and improve training stability. Our challenge submission achieves 22.569 PSNR, 0.675 SSIM, and 8.067 $ΔE$ on the final hidden test set, significantly improving over the baseline and achieving the 3rd best SSIM and $ΔE$ among all challenge entries. Our code is available at github.com/nuniniyujin/Unpaired-ISP .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。