arXiv:2502.04318cs.CV2025-02被引 1

用分层隐特征外推实现六视角稀疏重建,快速还原全景场景。

sshELF: Single-Shot Hierarchical Extrapolation of Latent Features for 3D Reconstruction from Sparse-Views

  • 分两阶段设计:先生成虚拟视图,再解码3D结构,提升效率与模块性。
  • 仅需6个输入视角即可重建360度全景,真实数据上表现优异。
  • 适合需要快速建模、支持实时渲染的户外场景应用。

从稀疏的外向视角重建无界室外场景面临视图重叠极少的挑战。以往方法缺乏跨场景理解,且以基本体为中心的建模方式过度依赖局部特征弥补全局上下文缺失,导致未见区域模糊。本文提出sshELF,一种基于隐特征分层外推的单次快速重建流水线。核心思路是将信息外推与基本体解码分离,实现训练场景间结构模式的高效迁移。方法包括:(1) 学习跨场景先验以生成中间虚拟视图,外推至未观测区域;(2) 采用两阶段网络设计,分离虚拟视图生成与3D基本体解码,支持高效训练与模块化架构;(3) 集成预训练基础模型,联合推理隐特征与纹理,增强场景理解与泛化能力。sshELF可从六个稀疏输入视角重建360度场景,在合成与真实数据集上表现竞争。实验表明其能准确重建遮挡区域,支持实时渲染,并为下游任务提供丰富隐特征。代码将开源。

原文摘要 · Abstract (English)

Reconstructing unbounded outdoor scenes from sparse outward-facing views poses significant challenges due to minimal view overlap. Previous methods often lack cross-scene understanding and their primitive-centric formulations overload local features to compensate for missing global context, resulting in blurriness in unseen parts of the scene. We propose sshELF, a fast, single-shot pipeline for sparse-view 3D scene reconstruction via hierarchal extrapolation of latent features. Our key insights is that disentangling information extrapolation from primitive decoding allows efficient transfer of structural patterns across training scenes. Our method: (1) learns cross-scene priors to generate intermediate virtual views to extrapolate to unobserved regions, (2) offers a two-stage network design separating virtual view generation from 3D primitive decoding for efficient training and modular model design, and (3) integrates a pre-trained foundation model for joint inference of latent features and texture, improving scene understanding and generalization. sshELF can reconstruct 360 degree scenes from six sparse input views and achieves competitive results on synthetic and real-world datasets. We find that sshELF faithfully reconstructs occluded regions, supports real-time rendering, and provides rich latent features for downstream applications. The code will be released.

3D重建稀疏视图隐空间外推实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。