用稀疏视角实现高精度网格与逼真渲染,兼顾几何清晰与视觉真实。
MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views
- 将场景几何建模为图集图表,用2D高斯面元实时渲染。
- 仅需少量输入视角即达成顶尖重建质量与渲染逼真度。
- 适合需要显式几何结构的视觉、图形与机器人应用。
我们提出一种新型外观模型,可从稀疏视图样本中同时实现高质量3D表面网格重建与逼真新视角合成。核心思想是将场景几何表面建模为图集图表(Atlas of Charts),并使用2D高斯面元(Gaussian surfels)进行渲染(MAtCha Gaussians)。MAtCha通过预训练单目深度估计器提取高频表面细节,并利用高斯面元渲染进行优化。高斯面元动态附着于图表上,兼顾神经体积渲染的逼真感与网格模型的清晰几何结构,解决二者看似矛盾的目标。核心创新包括新型神经形变模型与结构损失函数,有效保留深度学习所得的精细表面细节,同时克服其固有的尺度模糊问题。大量实验验证表明,MAtCha在表面重建质量和视觉逼真度上达到当前最优水平,且所需输入视角数量和计算时间显著减少。我们认为MAtCha将成为视觉、图形与机器人领域中需要显式几何与逼真度融合应用的基础工具。项目主页:https://anttwo.github.io/matcha/
原文摘要 · Abstract (English)
We present a novel appearance model that simultaneously realizes explicit high-quality 3D surface mesh recovery and photorealistic novel view synthesis from sparse view samples. Our key idea is to model the underlying scene geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels (MAtCha Gaussians). MAtCha distills high-frequency scene surface details from an off-the-shelf monocular depth estimator and refines it through Gaussian surfel rendering. The Gaussian surfels are attached to the charts on the fly, satisfying photorealism of neural volumetric rendering and crisp geometry of a mesh model, i.e., two seemingly contradicting goals in a single model. At the core of MAtCha lies a novel neural deformation model and a structure loss that preserve the fine surface details distilled from learned monocular depths while addressing their fundamental scale ambiguities. Results of extensive experimental validation demonstrate MAtCha's state-of-the-art quality of surface reconstruction and photorealism on-par with top contenders but with dramatic reduction in the number of input views and computational time. We believe MAtCha will serve as a foundational tool for any visual application in vision, graphics, and robotics that require explicit geometry in addition to photorealism. Our project page is the following: https://anttwo.github.io/matcha/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。