arXiv:2607.17411cs.GRcs.LG2026-07

用视觉特征引导扩散模型,实现无需梯度的逆渲染优化

Feature-Guided Diffusion for Non-Differentiable Inverse Rendering

论文配图:Feature-Guided Diffusion for Non-Differentiable Inverse Rendering
图 1 · 摘自论文原文
  • 用ViT提取渲染图像的密集特征,替代传统标量损失
  • 扩散模型生成候选参数,结合CMA策略持续优化收敛
  • 适用于路径追踪、向量样条等多类逆渲染问题

逆渲染通常依赖可微分渲染器与梯度下降,需大量领域工程,且易陷入局部最优。无导数方法虽降低工程负担,但对初始值敏感。本文提出特征引导扩散演化(FIDE),一个全黑箱框架,无需梯度或特定初始化:仅将渲染器视为输出图像的黑盒函数。核心思想是特征引导:不将每个候选渲染结果压缩为标量损失,而是用视觉变压器(ViT)提取其密集视觉特征。基于这些特征训练扩散型候选参数生成模型,使网络利用视觉线索预测匹配目标图像的参数。随后,通过闭环的CMA进化策略对候选解进行精炼,随优化进程不断缩小提案区域。在路径追踪、向量样条、Voronoi着色器和机器人等多样逆问题上验证,特征引导显著提升收敛性,并可靠跳出梯度法停滞的局部极小点。

原文摘要 · Abstract (English)

Inverse rendering is traditionally solved via differentiable renderers and gradient descent, which requires substantial problem-specific engineering and is prone to getting stuck in local minima due to ambiguities. Derivative-free approaches alleviate engineering requirements, but often heavily depend on a good problem initialization. In this work, we propose Feature-Informed Diffusion Evolution (FIDE), a fully black-box framework that requires no gradients or specific initialization: the renderer is treated as an opaque function whose only requirement is to produce images. Our key insight is feature guiding: rather than reducing each candidate rendering to a scalar loss value, we use a Vision Transformer (ViT) to extract dense visual features from it. We subsequently use these features to train a diffusion-based candidate proposal model, allowing the network to use visual cues to predict parameters that would match the target image. The candidate solutions proposed by this diffusion model are then refined in a closed loop with a CMA evolution strategy, continuously narrowing the proposal region as optimization progresses. We validate across diverse inverse problems from path tracing, vector splines, Voronoi shaders, and robotics, and demonstrate that feature-guiding substantially improves convergence over scalar-loss baselines and reliably escapes local minima where gradient-based methods stall.

逆渲染扩散模型特征引导黑箱优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。