arXiv:2511.22704cs.CV2025-11AAAI被引 1

无需密集视图,用自监督点图实现人中心场景的高效自由视角渲染。

Splat-SAP: Feed-Forward Gaussian Splatting for Human-Centered Scene with Scale-Aware Point Map Reconstruction

  • 通过像素级点图重建几何,独立建模每张图像以应对大稀疏性。
  • 两阶段训练:自监督学习点图尺度,再用光度损失优化立体匹配与渲染。
  • 适用于手持双目相机采集的稀疏人中心场景,渲染更稳定、视觉质量更高。

我们提出Splat-SAP,一种针对大稀疏性双目相机采集的人中心场景的前馈式渲染方法。高斯点绘(Gaussian Splatting)虽在渲染任务中表现优异,但通常需对每个场景进行密集输入视图的优化。尽管近期方法借助多视图立体(MVS)获取几何先验实现前馈渲染,仍需高度重叠的输入视图建立先验。为解决此问题,我们采用像素级点图表示几何,其独立建模各视角,对大稀疏性具有鲁棒性。整体采用两阶段学习策略:第一阶段通过迭代相似性学习将点图映射至真实空间,便于后续相机控制;第二阶段将双输入视图点图投影至目标视图平面,并通过立体匹配优化几何结构。最终在该精炼平面上锚定高斯原型以生成高质量图像。第一阶段的尺度感知点图为自监督学习,无需3D标注;第二阶段使用光度损失监督。我们收集了多视角人中心数据集,实验表明本方法显著提升了点图重建稳定性与自由视角渲染的视觉质量。

原文摘要 · Abstract (English)

We present Splat-SAP, a feed-forward approach to render novel views of human-centered scenes from binocular cameras with large sparsity. Gaussian Splatting has shown its promising potential in rendering tasks, but it typically necessitates per-scene optimization with dense input views. Although some recent approaches achieve feed-forward Gaussian Splatting rendering through geometry priors obtained by multi-view stereo, such approaches still require largely overlapped input views to establish the geometry prior. To bridge this gap, we leverage pixel-wise point map reconstruction to represent geometry which is robust to large sparsity for its independent view modeling. In general, we propose a two-stage learning strategy. In stage 1, we transform the point map into real space via an iterative affinity learning process, which facilitates camera control in the following. In stage 2, we project point maps of two input views onto the target view plane and refine such geometry via stereo matching. Furthermore, we anchor Gaussian primitives on this refined plane in order to render high-quality images. As a metric representation, the scale-aware point map in stage 1 is trained in a self-supervised manner without 3D supervision and stage 2 is supervised with photo-metric loss. We collect multi-view human-centered data and demonstrate that our method improves both the stability of point map reconstruction and the visual quality of free-viewpoint rendering.

三维重建高斯点绘自由视角稀疏输入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。