arXiv:2410.23742cs.CV2024-10中稿 · ICLR

共享基础平面,让千个物体用更少资源高效建模

Fused-Planes: Why Train a Thousand Tri-Planes When You Can Share?

  • 用共享基础平面+潜空间捕捉物体间结构共性
  • 训练速度提升7.2倍,内存降低3.2倍,质量不变
  • 适合大规模3D物体建模,尤其资源受限场景

三平面神经辐射场(Tri-Planar NeRFs)通过二维平面结构表示三维物体,使强大二维视觉模型可应用于三维任务,已成为大规模三维物体建模的主流方法。然而,现有方法为每个物体独立训练一个三平面,忽视了同类物体间的结构相似性,导致计算效率低下。为此,我们提出Fused-Planes,一种新型对象表示方法,在保持相同平面结构的前提下,通过潜空间和一组全局共享的基础平面显式捕捉物体间的结构共性。每个具体对象的表示由这些基础平面分解并辅以对象特异性特征构成。Fused-Planes在平面表示中达到顶尖效率:相比三平面,训练速度提升7.2倍,内存占用降低3.2倍,且渲染质量相当。其超轻量变体进一步将单个对象内存使用减少1875倍,质量损失极小。项目主页见 https://fused-planes.github.io。

原文摘要 · Abstract (English)

Tri-Planar NeRFs enable the application of powerful 2D vision models for 3D tasks, by representing 3D objects using 2D planar structures. This has made them the prevailing choice to model large collections of 3D objects. However, training Tri-Planes to model such large collections is computationally intensive and remains largely inefficient. This is because the current approaches independently train one Tri-Plane per object, hence overlooking structural similarities in large classes of objects. In response to this issue, we introduce Fused-Planes, a novel object representation that improves the resource efficiency of Tri-Planes when reconstructing object classes, all while retaining the same planar structure. Our approach explicitly captures structural similarities across objects through a latent space and a set of globally shared base planes. Each individual Fused-Planes is then represented as a decomposition over these base planes, augmented with object-specific features. Fused-Planes showcase state-of-the-art efficiency among planar representations, demonstrating $7.2 \times$ faster training and $3.2 \times$ lower memory footprint than Tri-Planes while maintaining rendering quality. An ultra-lightweight variant further cuts per-object memory usage by $1875 \times$ with minimal quality loss. Our project page can be found at https://fused-planes.github.io .

3D建模神经渲染高效表示共享平面

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。