arXiv:2410.01535cs.CV2024-10ICLR被引 12

用几何体与高斯点结合,实现可编辑的3D场景重建。

GaussianBlock: Building Part-Aware Compositional and Editable 3D Scene by Primitives and Gaussians

  • 用注意力引导的中心化损失+动态拆分融合,让几何体语义清晰。
  • 混合使用几何体与3D高斯点,保持高保真度同时支持精准编辑。
  • 适合需要可组合、可修改3D内容的研究者和开发者。

随着神经辐射场和高斯溅射的发展,3D重建技术已达到极高保真度。然而,这些方法学习到的隐式表示高度耦合且缺乏可解释性。本文提出一种新型部件感知的组合式重建方法GaussianBlock,实现语义一致且解耦的表示,支持类似积木般的精确物理编辑,同时保持高保真度。GaussianBlock采用混合表示:利用几何体的灵活操作性与可编辑性,结合3D高斯点在重建质量上的优势。通过基于2D语义先验的注意力引导中心化损失,实现语义一致的几何体生成,并辅以动态拆分与融合策略;3D高斯点与几何体融合,用于细化结构细节并提升保真度。此外,引入绑定继承策略强化二者关联。在多个基准测试中,重建场景表现出良好的解耦性、组合性与紧凑性,支持无缝、直接、精确的编辑,同时保持高质量。

原文摘要 · Abstract (English)

Recently, with the development of Neural Radiance Fields and Gaussian Splatting, 3D reconstruction techniques have achieved remarkably high fidelity. However, the latent representations learnt by these methods are highly entangled and lack interpretability. In this paper, we propose a novel part-aware compositional reconstruction method, called GaussianBlock, that enables semantically coherent and disentangled representations, allowing for precise and physical editing akin to building blocks, while simultaneously maintaining high fidelity. Our GaussianBlock introduces a hybrid representation that leverages the advantages of both primitives, known for their flexible actionability and editability, and 3D Gaussians, which excel in reconstruction quality. Specifically, we achieve semantically coherent primitives through a novel attention-guided centering loss derived from 2D semantic priors, complemented by a dynamic splitting and fusion strategy. Furthermore, we utilize 3D Gaussians that hybridize with primitives to refine structural details and enhance fidelity. Additionally, a binding inheritance strategy is employed to strengthen and maintain the connection between the two. Our reconstructed scenes are evidenced to be disentangled, compositional, and compact across diverse benchmarks, enabling seamless, direct and precise editing while maintaining high quality.

3D重建可编辑高斯溅射几何体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。