无需人工标注,实现动态驾驶场景的实例级3D重建
IDSplat: Instance-Decomposed 3D Gaussian Splatting for Driving Scenes
- 将动态物体建模为刚性变换的连贯实例,而非无结构时变体
- 在Waymo数据集上达到高质量重建,且实例分解效果稳定
- 适合大规模自动驾驶仿真,支持不同视角密度泛化
重建动态驾驶场景对发展基于传感器真实感仿真的自动驾驶系统至关重要。现有方法虽能实现高保真重建,但或依赖昂贵的人工标注获取物体轨迹,或使用无显式实例分解的时间变化表示,导致静态与动态元素纠缠,阻碍场景分离。我们提出IDSplat,一种自监督的3D高斯泼溅框架,可在无需人工标注的情况下实现具有显式实例分解和可学习运动轨迹的动态场景重建。核心思路是将动态物体建模为经历刚性变换的连贯实例,而非无结构的时间变化基元。通过零样本、语言引导的视频追踪,锚定于激光雷达的3D空间,并利用特征对应估计一致姿态。引入协同转向平滑方案,获得时空与物理一致的运动轨迹,缓解姿态错位与追踪失败问题,随后联合优化物体姿态与高斯参数。在Waymo Open Dataset上的实验表明,本方法在保持实例级分解的同时,实现了具有竞争力的重建质量,并能在不重新训练的情况下跨多样序列和视图密度泛化,适用于大规模自动驾驶应用。代码将公开。
原文摘要 · Abstract (English)
Reconstructing dynamic driving scenes is essential for developing autonomous systems through sensor-realistic simulation. Although recent methods achieve high-fidelity reconstructions, they either rely on costly human annotations for object trajectories or use time-varying representations without explicit object-level decomposition, leading to intertwined static and dynamic elements that hinder scene separation. We present IDSplat, a self-supervised 3D Gaussian Splatting framework that reconstructs dynamic scenes with explicit instance decomposition and learnable motion trajectories, without requiring human annotations. Our key insight is to model dynamic objects as coherent instances undergoing rigid transformations, rather than unstructured time-varying primitives. For instance decomposition, we employ zero-shot, language-grounded video tracking anchored to 3D using lidar, and estimate consistent poses via feature correspondences. We introduce a coordinated-turn smoothing scheme to obtain temporally and physically consistent motion trajectories, mitigating pose misalignments and tracking failures, followed by joint optimization of object poses and Gaussian parameters. Experiments on the Waymo Open Dataset demonstrate that our method achieves competitive reconstruction quality while maintaining instance-level decomposition and generalizes across diverse sequences and view densities without retraining, making it practical for large-scale autonomous driving applications. Code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。