相机内置AI会生成虚假细节,该文提出可逆方法还原真实图像。
Addressing Image Authenticity When Cameras Use Generative AI

- 用自编码器+MLP模型还原相机拍摄时未被AI修改的原始图像
- 仅需180KB存储空间,能作为元数据嵌入JPEG/HEIC格式
- 适合关注图像真实性、尤其在低光或变焦场景下的用户
生成式AI(GenAI)能够以逼真方式修改相机拍摄的图像,引发对网络图像真实性的担忧。尽管直接由相机捕获的图像通常被视为真实,但随着深度学习模块越来越多集成到相机的图像信号处理器(ISP)中,捕获时就可能引入幻觉内容。这类幻觉通常表现为边缘增强或纹理美化,但在基于AI的数字变焦或低光增强等操作中,可能改变图像语义和解读。因此,用户可能并未意识到照片内容已失真。本文提出一种方法,通过优化图像特定的多层感知机(MLP)解码器与模态专用编码器,从已含幻觉的相机图像中恢复出‘无幻觉’版本。该编码器与MLP为自包含结构,可在拍摄后独立使用,无需访问相机ISP。其总存储仅需180 KB,可轻松作为元数据保存于JPEG和HEIC等标准图像格式中。
原文摘要 · Abstract (English)
The ability of generative AI (GenAI) methods to photorealistically alter camera images has raised awareness about the authenticity of images shared online. Interestingly, images captured directly by our cameras are considered authentic and faithful. However, with the increasing integration of deep-learning modules into cameras' capture-time hardware -- namely, the image signal processor (ISP) -- there is now a potential for hallucinated content in images directly output by our cameras. Hallucinated capture-time image content is typically benign, such as enhanced edges or texture, but in certain operations, such as AI-based digital zoom or low-light image enhancement, hallucinations can potentially alter the semantics and interpretation of the image content. As a result, users may not realize that the content in their camera images is not authentic. This paper addresses this issue by enabling users to recover the 'unhallucinated' version of the camera image to avoid misinterpretation of the image content. Our approach works by optimizing an image-specific multi-layer perceptron (MLP) decoder together with a modality-specific encoder so that, given the camera image, we can recover the image before hallucinated content was added. The encoder and MLP are self-contained and can be applied post-capture to the image without requiring access to the camera ISP. Moreover, the encoder and MLP decoder require only 180 KB of storage and can be readily saved as metadata within standard image formats such as JPEG and HEIC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。