用神经纹理表面元实现稀疏几何下的实时新视角合成
Nexels: Neurally-Textured Surfels for Real-Time Novel View Synthesis with Sparse Geometries
- 用表面元表示几何，神经场+颜色编码外观，解耦建模
- 户外场景用9.7倍少的元素、5.5倍少内存，室内场景更优
- 比现有方法快一倍且画质更好，适合实时应用
尽管3D高斯点阵在新视角合成中表现优异，但即使场景几何简单，仍需数百万个基元来建模高纹理场景。本文提出一种超越点基表示的新方法，通过解耦几何与外观实现紧凑表示：使用表面元（surfels）表示几何，结合全局神经场和每个基元的颜色编码外观。神经场为每像素固定数量的基元提供纹理，计算开销低。该方法在户外场景下达到与3D高斯点阵相当的感知质量，但仅需其9.7倍少的基元和5.5倍少的内存；在室内场景下，基元减少31倍，内存降低3.7倍。同时渲染速度是现有纹理基元的两倍，且视觉质量更优。
原文摘要 · Abstract (English)
Though Gaussian splatting has achieved impressive results in novel view synthesis, it requires millions of primitives to model highly textured scenes, even when the geometry of the scene is simple. We propose a representation that goes beyond point-based rendering and decouples geometry and appearance in order to achieve a compact representation. We use surfels for geometry and a combination of a global neural field and per-primitive colours for appearance. The neural field textures a fixed number of primitives for each pixel, ensuring that the added compute is low. Our representation matches the perceptual quality of 3D Gaussian splatting while using $9.7\times$ fewer primitives and $5.5\times$ less memory on outdoor scenes and using $31\times$ fewer primitives and $3.7\times$ less memory on indoor scenes. Our representation also renders twice as fast as existing textured primitives while improving upon their visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。