用涂鸦控制单图室内光照,实现精准光影重渲染。
ScribbleLight: Single Image Indoor Relighting with Scribbles

- 基于涂鸦和法线图的ControlNet结构,实现光照局部控制。
- 通过反照率条件扩散模型,保持材质颜色纹理不变。
- 适合室内设计、虚拟看房等需精细调光的场景。
基于图像的室内重光照可增强空间的沉浸感,广泛应用于室内设计、虚拟布景和房地产展示。从单张图像实现室内重光照极具挑战性,源于多光源与杂乱物体间复杂的光照交互,以及几何与材质多样性。近年来,生成模型已成功用于图像重光照,但缺乏对局部光照的精细控制。本文提出ScribbleLight,一种支持通过涂鸦描述光照变化的生成模型。关键技术在于:基于反照率条件的Stable Image Diffusion模型,确保重光照后原图固有颜色与纹理不变;以及基于编码器-解码器的ControlNet架构,结合法线图与涂鸦标注,实现几何保持的光照效果。我们验证了ScribbleLight仅需稀疏涂鸦即可生成多种光照效果,如开关灯、添加高光、投射阴影或来自未见光源的间接照明。
原文摘要 · Abstract (English)
Image-based relighting of indoor rooms creates an immersive virtual understanding of the space, which is useful for interior design, virtual staging, and real estate. Relighting indoor rooms from a single image is especially challenging due to complex illumination interactions between multiple lights and cluttered objects featuring a large variety in geometrical and material complexity. Recently, generative models have been successfully applied to image-based relighting conditioned on a target image or a latent code, albeit without detailed local lighting control. In this paper, we introduce ScribbleLight, a generative model that supports local fine-grained control of lighting effects through scribbles that describe changes in lighting. Our key technical novelty is an Albedo-conditioned Stable Image Diffusion model that preserves the intrinsic color and texture of the original image after relighting and an encoder-decoder-based ControlNet architecture that enables geometry-preserving lighting effects with normal map and scribble annotations. We demonstrate ScribbleLight's ability to create different lighting effects (e.g., turning lights on/off, adding highlights, cast shadows, or indirect lighting from unseen lights) from sparse scribble annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。