arXiv:2505.10420cs.CVcs.AI2025-05CVPR被引 7

无需配对数据,用对抗训练实现手机图像处理的轻量化学习。

Learned Lightweight Smartphone ISP with Unpaired Data

  • 采用多判别器对抗损失,无需原始图与参考图像素对齐。
  • 在Zurich和Fujifilm数据集上达到接近有监督方法的图像保真度。
  • 适配移动端轻量模型,适合实际手机端部署应用。

图像信号处理器(ISP)是现代智能手机相机的核心组件,负责将传感器捕获的RAW数据转换为感知质量高的RGB图像。近期研究显示,深度学习方法能捕捉细节,其画质已接近专业相机水平。然而,训练学习型ISP的关键难点在于获取像素级对齐的成对数据——即同一场景的RAW图像与高质量参考图像。本文提出一种新型无配对训练方法,无需原始图像与目标图像的内容匹配即可学习ISP。该方法通过多判别器对抗训练,利用预训练网络提取的特征图构建多分支损失函数,在保持内容结构的同时,从目标RGB数据集中学习颜色与纹理特性。我们使用适用于移动设备的轻量级神经网络架构,在Zurich RAW to RGB和Fujifilm UltraISP数据集上验证了该方法。相比传统配对训练,本方法展现出强大潜力,在多个评估指标上均达到高保真效果。代码与预训练模型已开源。

原文摘要 · Abstract (English)

The Image Signal Processor (ISP) is a fundamental component in modern smartphone cameras responsible for conversion of RAW sensor image data to RGB images with a strong focus on perceptual quality. Recent work highlights the potential of deep learning approaches and their ability to capture details with a quality increasingly close to that of professional cameras. A difficult and costly step when developing a learned ISP is the acquisition of pixel-wise aligned paired data that maps the raw captured by a smartphone camera sensor to high-quality reference images. In this work, we address this challenge by proposing a novel training method for a learnable ISP that eliminates the need for direct correspondences between raw images and ground-truth data with matching content. Our unpaired approach employs a multi-term loss function guided by adversarial training with multiple discriminators processing feature maps from pre-trained networks to maintain content structure while learning color and texture characteristics from the target RGB dataset. Using lightweight neural network architectures suitable for mobile devices as backbones, we evaluated our method on the Zurich RAW to RGB and Fujifilm UltraISP datasets. Compared to paired training methods, our unpaired learning strategy shows strong potential and achieves high fidelity across multiple evaluation metrics. The code and pre-trained models are available at https://github.com/AndreiiArhire/Learned-Lightweight-Smartphone-ISP-with-Unpaired-Data .

ISP无监督学习轻量化手机摄影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。