一次前向传播完成动态场景的点跟踪与三维重建
DePT3R: Joint Dense Point Tracking and 3D Reconstruction of Dynamic Scenes in a Single Forward Pass
- 用统一框架同时实现点跟踪和3D重建，无需逐帧处理
- 在多个动态场景数据集上表现优异，内存效率显著提升
- 不依赖相机位姿，适合快速变化的复杂环境
现有动态场景稠密3D点跟踪方法通常依赖成对处理、已知相机位姿或输入帧的时间顺序，限制了灵活性与适用性。近期研究已实现从大规模无位姿图像集合中高效重建3D场景，为统一理解动态场景提供了契机。为此，我们提出DePT3R，一种在单次前向传播中同时完成多视角图像下动态场景稠密点跟踪与3D重建的新框架。该方法通过强大骨干网络提取时空特征，并使用密集预测头回归像素级映射图。关键优势在于无需相机位姿信息，大幅提升了在快速变化动态环境中的适应性与效率。我们在多个挑战性动态场景基准上验证了DePT3R，结果表明其性能强劲，且相比现有最先进方法在内存效率上显著提升。代码与数据已开源：https://github.com/StructuresComp/DePT3R
原文摘要 · Abstract (English)
Current methods for dense 3D point tracking in dynamic scenes typically rely on pairwise processing, require known camera poses, or assume temporal ordering of input frames, thereby constraining their flexibility and applicability. Additionally, recent advances have successfully enabled efficient 3D reconstruction from large-scale, unposed image collections, underscoring opportunities for unified approaches to dynamic scene understanding. Motivated by this, we propose DePT3R, a novel framework that simultaneously performs dense point tracking and 3D reconstruction of dynamic scenes from multiple images in a single forward pass. This multi-task learning is achieved by extracting deep spatio-temporal features with a powerful backbone and regressing pixel-wise maps with dense prediction heads. Crucially, DePT3R operates without requiring camera poses, substantially enhancing its adaptability and efficiency, especially important in dynamic environments with rapid changes. We validate DePT3R on several challenging benchmarks involving dynamic scenes, demonstrating strong performance and significant improvements in memory efficiency over existing state-of-the-art methods. Data and codes are available via the open repository: https://github.com/StructuresComp/DePT3R
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。