arXiv:2512.05635cs.CV2025-12被引 1

用无监督最优传输训练图像信号处理,不依赖配对数据也能达到顶尖效果。

Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Data

  • 基于非平衡最优传输,统一处理有/无配对数据的ISP训练
  • 无配对模式下性能媲美甚至超过原配对训练模型
  • 专家判别器团队精准修复颜色、结构和频域失真问题

学习型图像信号处理(ISP)管线虽具强大端到端性能,但严重依赖大规模原始数据到sRGB的配对数据集。这一高昂成本成为主要瓶颈。为此,我们提出一种基于最优传输的新型无监督训练框架,可同时在无配对和有配对模式下训练任意ISP架构。首次将非平衡最优传输(UOT)应用于此类跨域转换任务。该框架对目标sRGB数据中的异常值具有鲁棒性,可忽略难以映射的典型样本。核心创新是引入“专家判别器委员会”——一种混合对抗正则化器,通过提供针对性梯度,纠正颜色保真度、结构伪影及频域真实感等特定失败模式。实验表明:在配对模式下,本框架性能全面超越原有配对方法;而在无配对模式下,其定量与定性表现媲美甚至优于原配对训练模型。代码与预训练模型已开源:https://github.com/gosha20777/EGUOT-ISP.git。

原文摘要 · Abstract (English)

Learned Image Signal Processing (ISP) pipelines offer powerful end-to-end performance but are critically dependent on large-scale paired raw-to-sRGB datasets. This reliance on costly-to-acquire paired data remains a significant bottleneck. To address this challenge, we introduce a novel, unsupervised training framework based on Optimal Transport capable of training arbitrary ISP architectures in both unpaired and paired modes. We are the first to successfully apply Unbalanced Optimal Transport (UOT) for this complex, cross-domain translation task. Our UOT-based framework provides robustness to outliers in the target sRGB data, allowing it to discount atypical samples that would be prohibitively costly to map. A key component of our framework is a novel ``committee of expert discriminators,'' a hybrid adversarial regularizer. This committee guides the optimal transport mapping by providing specialized, targeted gradients to correct specific ISP failure modes, including color fidelity, structural artifacts, and frequency-domain realism. To demonstrate the superiority of our approach, we retrained existing state-of-the-art ISP architectures using our paired and unpaired setups. Our experiments show that while our framework, when trained in paired mode, exceeds the performance of the original paired methods across all metrics, our unpaired mode concurrently achieves quantitative and qualitative performance that rivals, and in some cases surpasses, the original paired-trained counterparts. The code and pre-trained models are available at: https://github.com/gosha20777/EGUOT-ISP.git.

ISP最优传输无监督学习图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。