用前馈网络直接从稀疏低分辨图像生成高分辨率3D高斯点云,提升重建质量与泛化能力。
SR3R: Rethinking Super-Resolution 3D Reconstruction With Feed-Forward Gaussian Splatting
- 提出前馈映射框架,跳过逐场景优化,直接从稀疏输入生成高分辨率3D高斯点云
- 在三个基准上超越现有方法,在未见场景上实现强零样本泛化性能
- 可插拔适配任意前馈3DGS模型,适合追求实时与跨场景通用性的应用
3D超分辨率(3DSR)旨在从多视角低分辨率(LR)图像重建高分辨率(HR)3D场景。现有方法依赖密集的低分辨率输入和逐场景优化,导致高频率先验仅继承自预训练2D超分辨率(2DSR)模型,严重限制重建保真度、跨场景泛化性及实时可用性。本文提出将3DSR重新定义为从稀疏低分辨率视图到高分辨率3D高斯点云(3DGS)表示的直接前馈映射,使模型能从大规模多场景数据中自主学习3D特异性高频几何与外观。为此,我们提出SR3R,一个直接通过学习映射网络从稀疏低分辨率视图预测高分辨率3DGS表示的前馈框架。为进一步提升重建保真度,引入高斯偏移学习与特征精炼,稳定重建过程并锐化高频细节。SR3R可插拔,兼容任意前馈3DGS重建主干:主干提供低分辨率3DGS骨架,SR3R将其升维至高分辨率3DGS。在三个3D基准上的大量实验表明,SR3R超越当前最优3DSR方法,并实现强大零样本泛化能力,甚至在未见场景上优于最先进逐场景优化方法。
原文摘要 · Abstract (English)
3D super-resolution (3DSR) aims to reconstruct high-resolution (HR) 3D scenes from low-resolution (LR) multi-view images. Existing methods rely on dense LR inputs and per-scene optimization, which restricts the high-frequency priors for constructing HR 3D Gaussian Splatting (3DGS) to those inherited from pretrained 2D super-resolution (2DSR) models. This severely limits reconstruction fidelity, cross-scene generalization, and real-time usability. We propose to reformulate 3DSR as a direct feed-forward mapping from sparse LR views to HR 3DGS representations, enabling the model to autonomously learn 3D-specific high-frequency geometry and appearance from large-scale, multi-scene data. This fundamentally changes how 3DSR acquires high-frequency knowledge and enables robust generalization to unseen scenes. Specifically, we introduce SR3R, a feed-forward framework that directly predicts HR 3DGS representations from sparse LR views via the learned mapping network. To further enhance reconstruction fidelity, we introduce Gaussian offset learning and feature refinement, which stabilize reconstruction and sharpen high-frequency details. SR3R is plug-and-play and can be paired with any feed-forward 3DGS reconstruction backbone: the backbone provides an LR 3DGS scaffold, and SR3R upscales it to an HR 3DGS. Extensive experiments across three 3D benchmarks demonstrate that SR3R surpasses state-of-the-art (SOTA) 3DSR methods and achieves strong zero-shot generalization, even outperforming SOTA per-scene optimization methods on unseen scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。