用视图间约束提升3D高斯渲染的细节与一致性
MVGSR: Multi-View Consistent 3D Gaussian Super-Resolution via Epipolar Guidance
- 基于相机位姿选择辅助视图,支持任意排列的多视角数据
- 首次引入对极线约束的多视角注意力机制,增强几何一致性
- 适用于物体和场景级重建,尤其适合无时间顺序的多视角数据
由低分辨率图像训练的3D高斯溅射(3DGS)重建场景不适合高分辨率渲染。因此需要一种3DGS超分辨率(SR)方法来连接低分辨率输入与高分辨率输出。早期方法依赖单图超分网络,缺乏视图间一致性且无法融合多视角互补信息。近期视频类方法虽尝试解决此问题,但需严格顺序帧,难以应用于非结构化多视角数据集。本文提出多视角一致3D高斯超分辨率(MVGSR),聚焦于融合多视角信息以实现具有高频细节和更高一致性的3DGS渲染。我们首先提出基于相机位姿的辅助视图选择方法,使方法可适配任意组织的多视角数据集,无需时间连续性或数据重排。此外,我们首次在3DGS SR中引入对极线约束的多视角注意力机制,作为所提多视角超分网络的核心,使模型能选择性聚合来自辅助视图的一致信息,提升3DGS表示的几何一致性和细节保真度。大量实验表明,该方法在物体级和场景级3DGS SR基准上均达到当前最优性能。
原文摘要 · Abstract (English)
Scenes reconstructed by 3D Gaussian Splatting (3DGS) trained on low-resolution (LR) images are unsuitable for high-resolution (HR) rendering. Consequently, a 3DGS super-resolution (SR) method is needed to bridge LR inputs and HR rendering. Early 3DGS SR methods rely on single-image SR networks, which lack cross-view consistency and fail to fuse complementary information across views. More recent video-based SR approaches attempt to address this limitation but require strictly sequential frames, limiting their applicability to unstructured multi-view datasets. In this work, we introduce Multi-View Consistent 3D Gaussian Splatting Super-Resolution (MVGSR), a framework that focuses on integrating multi-view information for 3DGS rendering with high-frequency details and enhanced consistency. We first propose an Auxiliary View Selection Method based on camera poses, making our method adaptable for arbitrarily organized multi-view datasets without the need of temporal continuity or data reordering. Furthermore, we introduce, for the first time, an epipolar-constrained multi-view attention mechanism into 3DGS SR, which serves as the core of our proposed multi-view SR network. This design enables the model to selectively aggregate consistent information from auxiliary views, enhancing the geometric consistency and detail fidelity of 3DGS representations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on both object-centric and scene-level 3DGS SR benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。