arXiv:2505.19813cs.CV2025-05CVPR被引 2

用全局上下文与局部几何融合,实现仅1-3视角的高质量新视图合成

GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis

  • 通过3D Transformer捕捉全局场景上下文,结合沿对极线的局部几何特征
  • 在1到3个输入视角下仍可生成高质量渲染结果,优于现有方法
  • 适合少样本新视图合成场景,尤其适用于数据受限的重建任务

神经辐射场(NeRF)通过直接从图像中建模场景特定的体素表示,革新了新视图合成。尽管通用的NeRF模型能通过学习潜在射线表示在未知场景中生成新视图,但其性能严重依赖大量多视角观测。当输入视角有限时,渲染质量显著下降。为此,我们提出GoLF-NRT:一种基于全局与局部特征融合的神经渲染变换器。该方法利用具有高效稀疏注意力的3D Transformer捕捉全局场景上下文,同时沿对极线提取局部几何特征,实现仅需1至3个输入视角即可高质量重建场景。此外,我们设计了一种基于注意力权重与核回归的自适应采样策略,提升基于Transformer的神经渲染精度。在多个公开数据集上的大量实验表明,GoLF-NRT在不同输入视角数量下均达到最先进性能,验证了方法的有效性与优越性。代码已开源:https://github.com/KLMAV-CUC/GoLF-NRT。

原文摘要 · Abstract (English)

Neural Radiance Fields (NeRF) have transformed novel view synthesis by modeling scene-specific volumetric representations directly from images. While generalizable NeRF models can generate novel views across unknown scenes by learning latent ray representations, their performance heavily depends on a large number of multi-view observations. However, with limited input views, these methods experience significant degradation in rendering quality. To address this limitation, we propose GoLF-NRT: a Global and Local feature Fusion-based Neural Rendering Transformer. GoLF-NRT enhances generalizable neural rendering from few input views by leveraging a 3D transformer with efficient sparse attention to capture global scene context. In parallel, it integrates local geometric features extracted along the epipolar line, enabling high-quality scene reconstruction from as few as 1 to 3 input views. Furthermore, we introduce an adaptive sampling strategy based on attention weights and kernel regression, improving the accuracy of transformer-based neural rendering. Extensive experiments on public datasets show that GoLF-NRT achieves state-of-the-art performance across varying numbers of input views, highlighting the effectiveness and superiority of our approach. Code is available at https://github.com/KLMAV-CUC/GoLF-NRT.

新视图合成少样本学习神经渲染3D重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。