用单一模型实现3D图像可控生成，支持多场景与视角切换。
CtrlNeRF: The Generative Neural Radiation Fields for the Controllable Synthesis of High-fidelity 3D-Aware Images
- 用共享权重的单个MLP建模多个场景，实现统一控制。
- 通过分离形状与外观代码，精准操控生成图像的3D结构和视觉效果。
- 支持训练外新视角合成，适合需要3D一致性的图像生成任务。
神经辐射场（NERF）通过多层感知机（MLP）学习3D几何的连续表示。将其融入生成模型后，生成神经辐射场（GRAF）可从随机噪声z生成图像，无需3D监督。实践中，形状与外观分别由代码z_s和z_a建模，实现推理时的独立操控。然而，单一MLP难以有效表征多个场景，且对3D几何的形状与外观控制精度不足。本文提出可控生成模型CtrlNeRF，采用单一MLP网络以共享权重表示多个场景，通过调节形状与外观代码，实现高保真、3D一致图像的可控生成。此外，该模型可通过相机位姿变换与特征插值，合成训练集中不存在的新视角。大量实验表明，其在3D感知图像生成方面优于现有方法。
原文摘要 · Abstract (English)
The neural radiance field (NERF) advocates learning the continuous representation of 3D geometry through a multilayer perceptron (MLP). By integrating this into a generative model, the generative neural radiance field (GRAF) is capable of producing images from random noise z without 3D supervision. In practice, the shape and appearance are modeled by z_s and z_a, respectively, to manipulate them separately during inference. However, it is challenging to represent multiple scenes using a solitary MLP and precisely control the generation of 3D geometry in terms of shape and appearance. In this paper, we introduce a controllable generative model (i.e. \textbf{CtrlNeRF}) that uses a single MLP network to represent multiple scenes with shared weights. Consequently, we manipulated the shape and appearance codes to realize the controllable generation of high-fidelity images with 3D consistency. Moreover, the model enables the synthesis of novel views that do not exist in the training sets via camera pose alteration and feature interpolation. Extensive experiments were conducted to demonstrate its superiority in 3D-aware image generation compared to its counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。