arXiv:2410.04402cs.CVcs.GR2024-10中稿 · ACM Multimedia 202…被引 4

用递归细分四面体实现可变形神经辐射场,兼顾渲染质量与操作灵活性。

Deformable NeRF using Recursively Subdivided Tetrahedra

  • 基于递归细分四面体网格,避免复杂重划分,实现多分辨率隐式编码。
  • 两阶段训练:先保留关键四面体,再精细优化物体细节,提升变形效果。
  • 适合需要高精度可变形3D重建的研究者,尤其关注真实场景应用。

尽管神经辐射场(NeRF)在新视角合成方面表现出色,但其隐式表示限制了对物体的显式操控。现有方法通过引入显式几何代理实现形变,但面临两大挑战:一是四面体剖分过程耗时且计算量大;二是复杂或细长结构常导致存储开销过大或网格质量差,影响形变能力。为此,我们提出DeformRF,将四面体网格的可操控性与特征网格的高质量渲染能力无缝结合。为避免劣质四面体和重复剖分,采用两阶段训练策略:初始使用近似规则四面体网格,保留物体周围的主干四面体;第二阶段通过更细粒度网格细化物体细节。同时引入递归细分四面体概念,隐式生成高分辨率网格,仅需存储第一阶段生成的粗粒度网格。我们在合成与真实采集数据集上进行全面评估,定量与定性结果均表明该方法在新视角合成与形变任务中有效。项目页面:https://ustc3dv.github.io/DeformRF/

原文摘要 · Abstract (English)

While neural radiance fields (NeRF) have shown promise in novel view synthesis, their implicit representation limits explicit control over object manipulation. Existing research has proposed the integration of explicit geometric proxies to enable deformation. However, these methods face two primary challenges: firstly, the time-consuming and computationally demanding tetrahedralization process; and secondly, handling complex or thin structures often leads to either excessive, storage-intensive tetrahedral meshes or poor-quality ones that impair deformation capabilities. To address these challenges, we propose DeformRF, a method that seamlessly integrates the manipulability of tetrahedral meshes with the high-quality rendering capabilities of feature grid representations. To avoid ill-shaped tetrahedra and tetrahedralization for each object, we propose a two-stage training strategy. Starting with an almost-regular tetrahedral grid, our model initially retains key tetrahedra surrounding the object and subsequently refines object details using finer-granularity mesh in the second stage. We also present the concept of recursively subdivided tetrahedra to create higher-resolution meshes implicitly. This enables multi-resolution encoding while only necessitating the storage of the coarse tetrahedral mesh generated in the first training stage. We conduct a comprehensive evaluation of our DeformRF on both synthetic and real-captured datasets. Both quantitative and qualitative results demonstrate the effectiveness of our method for novel view synthesis and deformation tasks. Project page: https://ustc3dv.github.io/DeformRF/

可变形三维重建神经辐射场四面体网格

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。