提出新型旋转等变编码器，更自然地建模球面光照分布
VENI: Variational Encoder for Natural Illumination
- 用新型向量神经网络与等变变换建模球面光照
- 在潜在空间实现更平滑的插值与更稳定结构
- 适合光照重建与生成任务的研究者参考
逆渲染是一个病态问题，但光照先验可帮助简化求解。现有方法或忽略光照环境的球面对称与旋转等变特性，或未提供良好的潜在空间。本文提出一种旋转等变变分自编码器，直接在球面上建模自然光照，无需依赖2D投影。为保持环境图的SO(2)等变性，采用新型向量神经视觉变压器（VN-ViT）作为编码器，并使用旋转等变条件神经场作为解码器。在编码器中，通过新颖的SO(2)等变全连接层将等变性从SO(3)降至SO(2)，该层是向量神经元的扩展。实验表明，该层在SO(2)等变模型中优于标准向量神经元。相比以往方法，本模型实现了更平滑的潜在空间插值，且具有更优良的潜在结构。
原文摘要 · Abstract (English)
Inverse rendering is an ill-posed problem, but priors such as illumination priors can help simplify it. Existing work either disregards the spherical and rotation-equivariant nature of illumination environments or does not provide a well-behaved latent space. We propose a rotation-equivariant variational autoencoder that models natural illumination on the sphere without relying on 2D projections. To preserve the SO(2)-equivariance of environment maps, we use a novel Vector Neuron Vision Transformer (VN-ViT) as encoder and a rotation-equivariant conditional neural field as decoder. In the encoder, we reduce the equivariance from SO(3) to SO(2) using a novel SO(2)-equivariant fully connected layer, an extension of Vector Neurons. We show that our SO(2)-equivariant fully connected layer outperforms standard Vector Neurons when used in our SO(2)-equivariant model. Compared to previous methods, our variational autoencoder enables smoother interpolation in latent space and offers a more well-behaved latent space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。