用解耦网格隐式场实现高效多功能3D编辑
NeuMesh++: Towards Versatile and Efficient Volumetric Editing with Disentangled Neural Mesh-based Implicit Field
- 将几何、纹理、语义编码解耦到网格顶点上，支持多类编辑
- 支持精准纹理填充/替换与语义引导编辑，提升细节保真度
- 适合需要精细3D内容编辑的研究者和工业应用
近期神经隐式渲染技术在新视角合成与3D场景重建中发展迅速，但现有编辑方法功能有限，如仅支持刚性变换或类别特定编辑。本文提出一种基于网格的新型表示方法，将神经辐射场在网格顶点上解耦为几何、纹理和语义代码，实现了多种高效且全面的编辑功能，包括网格引导的几何编辑、指定纹理编辑（含纹理替换、填充与绘画）、以及语义引导编辑。为此，我们设计了局部空间参数化以提升渲染质量与训练稳定性，引入可学习的顶点修改颜色以增强纹理编辑保真度，采用空间感知优化策略实现精确纹理编辑，并通过语义辅助区域选择减少隐式场编辑中的标注负担。在真实与合成数据集上的大量实验与编辑示例验证了本方法在表示质量和编辑能力上的优越性。
原文摘要 · Abstract (English)
Recently neural implicit rendering techniques have evolved rapidly and demonstrated significant advantages in novel view synthesis and 3D scene reconstruction. However, existing neural rendering methods for editing purposes offer limited functionalities, e.g., rigid transformation and category-specific editing. In this paper, we present a novel mesh-based representation by encoding the neural radiance field with disentangled geometry, texture, and semantic codes on mesh vertices, which empowers a set of efficient and comprehensive editing functionalities, including mesh-guided geometry editing, designated texture editing with texture swapping, filling and painting operations, and semantic-guided editing. To this end, we develop several techniques including a novel local space parameterization to enhance rendering quality and training stability, a learnable modification color on vertex to improve the fidelity of texture editing, a spatial-aware optimization strategy to realize precise texture editing, and a semantic-aided region selection to ease the laborious annotation of implicit field editing. Extensive experiments and editing examples on both real and synthetic datasets demonstrate the superiority of our method on representation quality and editing ability. Project page: https://zju3dv.github.io/neumeshplusplus/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。