用NeRF建模3D纹理,让草叶布料等复杂结构更真实。
NeRF-Texture: Synthesizing Neural Radiance Field Textures
- 将3D纹理分解为几何基形与隐含纹理特征,实现解耦建模
- 通过隐空间聚类约束提升特征匹配质量,合成更一致的纹理
- 支持平面与曲面纹理生成,适用于真实场景应用
纹理合成是计算机图形学中的基础问题,对多种应用具有重要意义。现有方法在处理2D图像纹理方面表现良好,但无法有效建模草、树叶、织物等含有3D几何尺度结构的真实纹理。本文提出一种基于神经辐射场(NeRF)的新纹理合成方法,从多视角图像中捕捉并生成纹理。所提出的NeRF纹理表示将具有精细几何细节的场景解耦为中尺度结构纹理和底层基形,使具有中尺度结构的纹理可作为位于基形上的隐含特征被有效学习,并输入至同步训练的NeRF解码器以表示丰富的视角依赖外观。利用这种隐式表示,我们通过隐特征块匹配方式合成基于NeRF的纹理。然而,重建内容空间与隐特征空间之间的度量不一致性可能影响合成质量。为此,我们进一步引入聚类约束来正则化隐特征分布,以提升匹配性能。实验结果表明,该方法不仅能生成平面域上的NeRF纹理,还能在曲面上合成实用的NeRF纹理,效果显著优于现有方法。
原文摘要 · Abstract (English)
Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。