让自动驾驶场景中的物体各自独立重建,支持动态编辑与理解。
InstDrive: Instance-Aware 3D Gaussian Splatting for Driving Scenes
- 用SAM生成掩码引导2D特征学习,结合对比损失和伪监督。
- 3D引入体素损失与轻量静态代码本,实现实例身份隐式编码。
- 首个在开放道路场景中实现3D实例分割的动态重建框架。
从行车记录仪视频重建动态驾驶场景受到越来越多关注,对自动驾驶与场景理解具有重要意义。尽管近期进展显著,多数方法仍将所有背景元素统一表示,限制了实例级理解与灵活编辑能力。部分方法尝试将2D分割映射至3D空间,但通常依赖预处理的实例ID或复杂流程,将连续特征映射为离散身份。此外,这些方法多针对室内场景设计,视角丰富,难以适用于户外驾驶场景。本文提出InstDrive,一种面向动态驾驶场景交互式重建的实例感知3D高斯溅射框架。利用SAM生成的掩码作为伪真值,通过对比损失和伪监督目标指导2D特征学习。在3D层面,引入正则化以隐式编码实例身份,并通过体素损失保证一致性。一个轻量级静态代码本在不需数据预处理或复杂优化的前提下,连接连续特征与离散身份。定量与定性实验验证了InstDrive的有效性,据我们所知,它是首个在动态开放世界驾驶场景中实现3D实例分割的框架。
原文摘要 · Abstract (English)
Reconstructing dynamic driving scenes from dashcam videos has attracted increasing attention due to its significance in autonomous driving and scene understanding. While recent advances have made impressive progress, most methods still unify all background elements into a single representation, hindering both instance-level understanding and flexible scene editing. Some approaches attempt to lift 2D segmentation into 3D space, but often rely on pre-processed instance IDs or complex pipelines to map continuous features to discrete identities. Moreover, these methods are typically designed for indoor scenes with rich viewpoints, making them less applicable to outdoor driving scenarios. In this paper, we present InstDrive, an instance-aware 3D Gaussian Splatting framework tailored for the interactive reconstruction of dynamic driving scene. We use masks generated by SAM as pseudo ground-truth to guide 2D feature learning via contrastive loss and pseudo-supervised objectives. At the 3D level, we introduce regularization to implicitly encode instance identities and enforce consistency through a voxel-based loss. A lightweight static codebook further bridges continuous features and discrete identities without requiring data pre-processing or complex optimization. Quantitative and qualitative experiments demonstrate the effectiveness of InstDrive, and to the best of our knowledge, it is the first framework to achieve 3D instance segmentation in dynamic, open-world driving scenes.More visualizations are available at our project page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。