arXiv:2605.03463cs.CV2026-05

先学形状再学语义,高效实现室内三维重建

First Shape, Then Meaning: Efficient Geometry and Semantics Learning for Indoor Reconstruction

论文配图:First Shape, Then Meaning: Efficient Geometry and Semantics Learning for Indoor Reconstruction
图 1 · 摘自论文原文
  • 分两步优化:先用RGB和几何线索学形状,再估计语义场
  • 在Replica上训练快2.3倍,在ScanNet++上更抗真实场景噪声
  • 无需复杂模块,简化设计仍能更好恢复物体表面

神经表面重建已成为室内三维重建的标准方法,有符号距离函数(SDF)在表示场景几何方面尤为有效。多种应用需要对场景上下文进行细致理解,推动了对象级语义信号的需求。尽管近期方法成功整合了语义标签,但往往继承了多SDF学习的慢训练时间和有限可扩展性。本文提出FSTM,一种通过两阶段过程统一学习几何与语义的方法:首先使用RGB输入和几何线索进行几何预热,随后进行语义场估计。通过先在无语义监督下优化几何,相比标准联合优化取得显著提升。FSTM表明,无需专用模块或复杂多SDF设计,简洁公式即可实现强大的几何与语义重建。在合成与真实室内数据集上的实验显示,该方法优于多SDF方法,在Replica上训练快2.3倍,在ScanNet++上更具鲁棒性,并在更多物体表面恢复上获得更高召回率。

原文摘要 · Abstract (English)

Neural Surface Reconstruction has become a standard methodology for indoor 3D reconstruction, with Signed Distance Functions (SDFs) proving particularly effective for representing scene geometry. A variety of applications require a detailed understanding of the scene context, driving the need for object-level semantic signals. While recent methods successfully integrate semantic labels, they often inherit the slow training time and limited scalability of multi-SDF learning. In this paper, we introduce FSTM, a unified approach for learning geometry and semantics through a two-step process: a geometry warm-up using RGB inputs and geometric cues, followed by semantic field estimation. By first optimising geometry without semantic supervision, we observe substantial improvements compared to the standard joint optimisation. Rather than relying on specialised modules or complex multi-SDF designs, FSTM shows that a streamlined formulation is sufficient to achieve strong geometric and semantic reconstructions. Experiments on both synthetic and real-world indoor datasets show that our method outperforms multi-SDF approaches. It trains 2.3x faster on Replica, improves robustness to real-world imperfections on ScanNet++, and achieves higher recall by recovering the surfaces of more objects in the scene. The code will be made available at https://remichierchia.github.io/FSTM.

三维重建语义学习SDF高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。