arXiv:2503.14698cs.CV2025-03NeurIPS被引 5

提出融合精炼模块,提升3D高斯点云的效率与动态场景适应性。

Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting

  • 在3D空间中融合多视角像素对齐点云,通过体素层次聚合特征
  • 采用稀疏体素变换器生成优化后的高斯点,减少冗余并支持动态重建
  • 可在单张H100显卡上实现15帧/秒的实时交互重建

基于前馈式3D高斯点云的最新进展显著提升了从稀疏视角高效重建场景的能力。然而,现有方法通常将高斯原语直接对齐输入图像中的像素,导致视图重叠时产生冗余,并且原语位置被约束在输入射线上,缺乏在3D空间中的自由度。此外,这类像素对齐方法难以自然扩展至动态场景,因需有效利用时间信息以处理重复和新增内容。为此,我们提出一种新颖的融合-精炼模块,通过在规范3D空间中合并与优化原语,增强现有前馈模型。核心是高效的混合点云-体素表示:从初始像素对齐的高斯原语出发,将局部特征聚合到粗到细的体素层级,并使用稀疏体素变换器处理体素特征,生成优化后的高斯原语。该方法能将任意数量输入融合为一致的原语集合,有效减少冗余,自然适配时间帧,实现历史感知的在线动态场景重建。在静态与流式场景重建中均达到当前最优性能,且在单张H100 GPU上运行达15帧/秒(延迟350毫秒)。

原文摘要 · Abstract (English)

Recent advances in feed-forward 3D Gaussian Splatting have led to rapid improvements in efficient scene reconstruction from sparse views. However, most existing approaches construct Gaussian primitives directly aligned with the pixels in one or more of the input images. This leads to redundancies in the representation when input views overlap and constrains the position of the primitives to lie along the input rays without full flexibility in 3D space. Moreover, these pixel-aligned approaches do not naturally generalize to dynamic scenes, where effectively leveraging temporal information requires resolving both redundant and newly appearing content across frames. To address these limitations, we introduce a novel Fuse-and-Refine module that enhances existing feed-forward models by merging and refining the primitives in a canonical 3D space. At the core of our method is an efficient hybrid Splat-Voxel representation: from an initial set of pixel-aligned Gaussian primitives, we aggregate local features into a coarse-to-fine voxel hierarchy, and then use a sparse voxel transformer to process these voxel features and generate refined Gaussian primitives. By fusing and refining an arbitrary number of inputs into a consistent set of primitives, our representation effectively reduces redundancy and naturally adapts to temporal frames, enabling history-aware online reconstruction of dynamic scenes. Our approach achieves state-of-the-art performance in both static and streaming scene reconstructions while running at interactive rates (15 fps with 350ms delay) on a single H100 GPU.

3D重建高斯点云动态场景实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。