无需依赖视图间几何约束,实现更通用的3D新视角合成。
Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis
- 用跨视图自监督ViT增强多视角特征提取,避免依赖极线先验。
- 在RealEstate10K和ACID数据集上超越现有方法,重建精度更高。
- 适合需要快速泛化到新场景的视觉系统开发者使用。
通用3D高斯溅射(3DGS)可在前馈推理中从稀疏视角观测重建新场景,无需传统3DGS所需的场景特定重训练。然而,现有方法严重依赖极线先验,在复杂真实场景中尤其不可靠,特别是在非重叠和遮挡区域。本文提出eFreeSplat,一种基于3DGS的高效前馈通用新视角合成模型,完全摆脱极线约束。为提升多视角特征提取与三维感知能力,采用在大规模数据集上通过跨视图补全预训练的自监督视觉变换器(ViT)。同时引入迭代跨视图高斯对齐方法,确保不同视角间深度尺度一致。eFreeSplat创新性地通过跨视图预训练提供3D先验,聚焦于无极线约束的特征匹配与编码。在RealEstate10K和ACID数据集上的宽基线新视角合成任务评估显示,其性能优于依赖极线先验的最先进方法,实现了更优的几何重建与新视角合成质量。
原文摘要 · Abstract (English)
Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. However, existing methods rely heavily on epipolar priors, which can be unreliable in complex realworld scenes, particularly in non-overlapping and occluded regions. In this paper, we propose eFreeSplat, an efficient feed-forward 3DGS-based model for generalizable novel view synthesis that operates independently of epipolar line constraints. To enhance multiview feature extraction with 3D perception, we employ a selfsupervised Vision Transformer (ViT) with cross-view completion pre-training on large-scale datasets. Additionally, we introduce an Iterative Cross-view Gaussians Alignment method to ensure consistent depth scales across different views. Our eFreeSplat represents an innovative approach for generalizable novel view synthesis. Different from the existing pure geometry-free methods, eFreeSplat focuses more on achieving epipolar-free feature matching and encoding by providing 3D priors through cross-view pretraining. We evaluate eFreeSplat on wide-baseline novel view synthesis tasks using the RealEstate10K and ACID datasets. Extensive experiments demonstrate that eFreeSplat surpasses state-of-the-art baselines that rely on epipolar priors, achieving superior geometry reconstruction and novel view synthesis quality. Project page: https://tatakai1.github.io/efreesplat/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。