用2D追踪器替代3D追踪，实现更鲁棒的街景动态物体重建
Street Gaussians without 3D Object Tracker
- 基于2D深度追踪器关联信息，融合3D物体实现稳定追踪
- 在隐式特征空间学习运动，自动修正轨迹误差并补全丢失检测
- 在Waymo-NOTR和KITTI数据集上超越现有方法，适合自动驾驶场景
驾驶场景中的真实感场景重建因快速移动物体而面临挑战。现有方法多依赖人工标注物体姿态，在规范空间中重建动态物体并根据姿态渲染。部分方法尝试用3D物体追踪器替代人工标注，但受限于大规模3D数据集稀缺，3D追踪器泛化能力有限，导致真实环境重建效果不佳。相比之下，2D基础模型表现出更强泛化能力。为此，我们提出一种新方法：利用2D深度追踪器的关联信息，结合3D物体融合策略构建稳定物体追踪模块。针对不可避免的追踪误差，进一步在隐式特征空间引入运动学习策略，自主修正轨迹偏差并恢复遗漏检测。在Waymo-NOTR和KITTI数据集上的实验结果表明，该方法优于现有技术。代码将发布于https://lolrudy.github.io/No3DTrackSG/。
原文摘要 · Abstract (English)
Realistic scene reconstruction in driving scenarios poses significant challenges due to fast-moving objects. Most existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space and move them based on these poses during rendering. While some approaches attempt to use 3D object trackers to replace manual annotations, the limited generalization of 3D trackers -- caused by the scarcity of large-scale 3D datasets -- results in inferior reconstructions in real-world settings. In contrast, 2D foundation models demonstrate strong generalization capabilities. To eliminate the reliance on 3D trackers and enhance robustness across diverse environments, we propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy. We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections. Experimental results on Waymo-NOTR and KITTI show that our method outperforms existing approaches. Our code will be released on https://lolrudy.github.io/No3DTrackSG/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。