用扩散模型生成真实相机原始图像,支持文本控制与多设备适配。
RawGen: Learning Camera Raw Image Generation

- 基于扩散模型设计新框架,从sRGB反推物理一致的线性原始数据。
- 构建多源sRGB到同一场景的映射数据集,解决不同相机处理差异问题。
- 可生成可控原始图像,提升低层视觉任务的数据多样性与性能。
相机捕获的是场景相关的线性原始图像,经机内图像信号处理器(ISP)转换为显示相关的8位sRGB输出。尽管原始数据更适用于低层视觉任务,但大规模原始数据集的获取仍是主要瓶颈,现有数据集规模有限且依赖特定相机硬件。生成模型为缓解数据稀缺提供可能,但现有扩散框架主要用于生成已处理的sRGB图像,而非物理一致的线性表示。本文提出RawGen,据我们所知首个基于扩散模型实现任意目标相机文本到原始图像生成及sRGB到原始图像逆变换的框架。RawGen利用大规模sRGB扩散模型的生成先验,通过在潜在空间和像素空间的特殊处理,合成具有物理意义的线性输出(如CIE XYZ或相机特定原始表示)。为应对扩散模型训练数据中未知且多样的ISP流水线和后期处理效果,我们构建了一个多对一的逆ISP数据集:同一场景下使用不同ISP参数生成的多个sRGB版本均锚定于一个共同的场景相关目标。在该数据集上微调条件去噪器与专用解码器,使RawGen能获得相机中心化的线性重建结果,有效逆向渲染流程。实验表明,RawGen在性能上优于假设固定ISP的传统逆ISP方法。此外,我们还证明,将RawGen生成的可扩展、文本驱动的合成数据融入训练流程,可显著提升下游低层视觉任务的效果。
原文摘要 · Abstract (English)
Cameras capture scene-referred linear raw images, which are processed by onboard image signal processors (ISPs) into display-referred 8-bit sRGB outputs. Although raw data is more faithful for low-level vision tasks, collecting large-scale raw datasets remains a major bottleneck, as existing datasets are limited and tied to specific camera hardware. Generative models offer a promising way to address this scarcity -- however, existing diffusion frameworks are designed to synthesize photo-finished sRGB images rather than physically consistent linear representations. This paper presents RawGen, to our knowledge the first diffusion-based framework enabling text-to-raw generation for arbitrary target cameras, alongside sRGB-to-raw inversion. RawGen leverages the generative priors of large-scale sRGB diffusion models to synthesize physically meaningful linear outputs, such as CIE XYZ or camera-specific raw representations, via specialized processing in latent and pixel spaces. To handle unknown and diverse ISP pipelines and photo-finishing effects in diffusion-model training data, we build a many-to-one inverse-ISP dataset where multiple sRGB renditions of the same scene generated using diverse ISP parameters are anchored to a common scene-referred target. Fine-tuning a conditional denoiser and specialized decoder on this dataset allows RawGen to obtain camera-centric linear reconstructions that effectively invert the rendering pipeline. We demonstrate RawGen's superior performance over traditional inverse-ISP methods that assume a fixed ISP. Furthermore, we show that augmenting training pipelines with RawGen's scalable, text-driven synthetic data can benefit downstream low-level vision tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。