无需训练即可精确控制图像中物体的遮挡关系。
LaRender: Training-Free Occlusion Control in Image Generation via Latent Rendering
- 在隐空间用体渲染原理模拟场景,根据遮挡关系和物体透射率生成图像。
- 相比现有方法,遮挡准确性显著提升,且不需微调预训练模型。
- 可灵活调整物体透明度、密度、光照等效果,适合创意设计与视觉特效。
我们提出一种全新的无需训练的图像生成算法,可精确控制图像中物体间的遮挡关系。现有图像生成方法通常依赖提示词影响遮挡,精度不足;布局到图像的方法虽能控制物体位置,却无法显式处理遮挡关系。本方法基于预训练图像扩散模型,利用体渲染原理在隐空间“渲染”场景,受遮挡关系和物体透射率引导。该方法无需重训练或微调模型,凭借物理基础实现精准遮挡控制。大量实验表明,其在遮挡准确性上显著优于现有方法。此外,通过调整渲染时物体的不透明度或概念权重,可实现物体透明度变化、物体密度(如森林)、粒子浓度(如雨、雾)、光照强度及镜头效果等多种视觉效果。
原文摘要 · Abstract (English)
We propose a novel training-free image generation algorithm that precisely controls the occlusion relationships between objects in an image. Existing image generation methods typically rely on prompts to influence occlusion, which often lack precision. While layout-to-image methods provide control over object locations, they fail to address occlusion relationships explicitly. Given a pre-trained image diffusion model, our method leverages volume rendering principles to "render" the scene in latent space, guided by occlusion relationships and the estimated transmittance of objects. This approach does not require retraining or fine-tuning the image diffusion model, yet it enables accurate occlusion control due to its physics-grounded foundation. In extensive experiments, our method significantly outperforms existing approaches in terms of occlusion accuracy. Furthermore, we demonstrate that by adjusting the opacities of objects or concepts during rendering, our method can achieve a variety of effects, such as altering the transparency of objects, the density of mass (e.g., forests), the concentration of particles (e.g., rain, fog), the intensity of light, and the strength of lens effects, etc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。