针对稀疏视角下的目标重建,提出语义与深度引导的高斯点渲染方法。
TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views
- 用语义掩码和深度先验约束目标区域,聚焦资源重建特定物体。
- 通过身份编码与空间一致性正则化,实现3D高斯聚类并减少冗余。
- 在多个数据集上优于现有方法,尤其适合复杂结构目标的精细重建。
最近的高斯点渲染进展显著推动了3D场景的全景与交互式分割。然而,现有方法常忽视从稀疏视角重建特定目标的需求,尤其当目标结构复杂时。为此,我们提出TSGaussian,结合语义约束与深度先验,防止在新视角合成中出现几何退化。该方法优先分配计算资源于指定目标,同时最小化背景开销。YOLOv9生成的边界框作为提示,驱动Segment Anything Model生成2D掩码,确保语义准确性和成本效率。TSGaussian引入紧凑的身份编码对每个高斯椭球进行区分,并加入3D空间一致性正则化以实现有效聚类。基于此,设计了一种剪枝策略,显著降低3D高斯冗余。大量实验表明,TSGaussian在三个标准数据集及自建挑战性数据集上均优于当前最优方法,在特定物体的新视角合成任务中表现卓越。代码已开源:https://github.com/leon2000-ai/TSGaussian。
原文摘要 · Abstract (English)
Recent advances in Gaussian Splatting have significantly advanced the field, achieving both panoptic and interactive segmentation of 3D scenes. However, existing methodologies often overlook the critical need for reconstructing specified targets with complex structures from sparse views. To address this issue, we introduce TSGaussian, a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in challenging novel view synthesis tasks. Our approach prioritizes computational resources on designated targets while minimizing background allocation. Bounding boxes from YOLOv9 serve as prompts for Segment Anything Model to generate 2D mask predictions, ensuring semantic accuracy and cost efficiency. TSGaussian effectively clusters 3D gaussians by introducing a compact identity encoding for each Gaussian ellipsoid and incorporating 3D spatial consistency regularization. Leveraging these modules, we propose a pruning strategy to effectively reduce redundancy in 3D gaussians. Extensive experiments demonstrate that TSGaussian outperforms state-of-the-art methods on three standard datasets and a new challenging dataset we collected, achieving superior results in novel view synthesis of specific objects. Code is available at: https://github.com/leon2000-ai/TSGaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。