Point4D可端到端重建数百帧长视频的3D运动轨迹，突破了以往方法只能处理几十帧的限制。
Point4D: Long-range 4D Motion Reconstruction
- 采用基于3D查询的运动解码器，将轨迹预测与可见性解耦，支持长序列连续推理。
- 在超过200帧的多类长视频追踪基准上达到当前最优性能，显著优于已有前馈式4D方法。
- 通过复用任意可见帧的视觉特征提升精度，不依赖源图像块，适合长时序动态场景建模。
我们提出Point4D，一种用于长时序视频序列4D重建的前馈模型。该模型能够可靠地推断跨数百帧视频中每个点的密集3D轨迹，而现有4D方法通常仅限于最多几十帧的短输入窗口。其关键创新在于灵活的3D查询式运动解码器，将轨迹预测与图像平面可见性解耦。预测出的3D终点在下一片段中直接重新查询，无需重投影或匹配。此外，我们发现从点可见的任意帧提取并复用视觉描述符，比仅依赖源图像块表现更优。总体而言，Point4D在跨越200帧以上的多种长视频追踪基准上实现当前最优性能，显著超越此前的前馈式4D方法。
原文摘要 · Abstract (English)
We introduce Point4D, a feed-forward model for 4D reconstruction of long-range video sequences. Point4D is able to reliably infer dense per-point 3D trajectories across multi-hundred-frame videos, unlike existing 4D methods that are limited to short input windows of at most a few dozen frames. A key innovation that enables this is our flexible 3D query-based motion decoder that decouples trajectory prediction from image-plane visibility. The predicted 3D endpoints are then directly re-queried in the next chunk without re-projection or matching. Furthermore, we show that extracting and reusing a visual descriptor from an arbitrary frame where the point is visible leads to better performance than relying solely on the source patch. Overall, Point4D achieves state-of-the-art performance across diverse long-video tracking benchmarks spanning over 200 frames and largely outperforms previous feed-forward 4D method. Project page: https://point-4d.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。