针对小基线视频的相机位姿估计,利用高斯点云渲染提升稳定性。
SmallGS: Gaussian Splatting-based Camera Pose Estimation for Small-Baseline Videos
- 基于高斯点云重建首帧场景,提供稳定参考帧
- 在小视角变化下实现比MonST3R更优的位姿精度
- 无需特征匹配,适合社交短视频等动态小运动场景
日常生活中广泛存在的小基线动态视频给现有位姿估计框架带来挑战,主要源于特征模糊、漂移累积及三角测量约束不足。高斯点云渲染在视角变化较小时能保持稳定的新视角合成能力。受此启发,我们提出SmallGS,专为小基线视频设计的相机位姿估计框架。SmallGS通过高斯点云优化连续相机位姿,从每段视频的第一帧重建场景作为稳定参考。在有限视角差异下,高斯点云的时间一致性降低了对深度变化的要求。我们进一步将预训练鲁棒视觉特征(如DINOv2)引入高斯点云中,通过高维特征图渲染增强位姿估计鲁棒性。通过冻结高斯点云并基于渲染特征优化相机视角,SmallGS无需显式特征对应或强视差运动即可有效学习位姿。我们在TUM-Dynamics数据集上验证了SmallGS的有效性,相比MonST3R和DORID-SLAM在动态场景的小基线视频中取得了显著更高的位姿估计精度。
原文摘要 · Abstract (English)
Dynamic videos with small baseline motions are ubiquitous in daily life, especially on social media. However, these videos present a challenge to existing pose estimation frameworks due to ambiguous features, drift accumulation, and insufficient triangulation constraints. Gaussian splatting, which maintains an explicit representation for scenes, provides a reliable novel view rasterization when the viewpoint change is small. Inspired by this, we propose SmallGS, a camera pose estimation framework that is specifically designed for small-baseline videos. SmallGS optimizes sequential camera poses using Gaussian splatting, which reconstructs the scene from the first frame in each video segment to provide a stable reference for the rest. The temporal consistency of Gaussian splatting within limited viewpoint differences reduced the requirement of sufficient depth variations in traditional camera pose estimation. We further incorporate pretrained robust visual features, e.g. DINOv2, into Gaussian splatting, where high-dimensional feature map rendering enhances the robustness of camera pose estimation. By freezing the Gaussian splatting and optimizing camera viewpoints based on rasterized features, SmallGS effectively learns camera poses without requiring explicit feature correspondences or strong parallax motion. We verify the effectiveness of SmallGS in small-baseline videos in TUM-Dynamics sequences, which achieves impressive accuracy in camera pose estimation compared to MonST3R and DORID-SLAM for small-baseline videos in dynamic scenes. Our project page is at: https://yuxinyao620.github.io/SmallGS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。