将2D图像信息高效映射回3D高斯点云，实现快速着色与语义分割。
Instant Colorization of Gaussian Splats
- 用法向方程求解可见性加权最小二乘问题，实现逆向映射。
- 在场景重光照、特征增强任务中提速达十倍以上。
- 适合需要快速3D着色或语义标注的实时应用。
高斯点阵最近成为生成逼真3D场景与渲染的主流框架。尽管现有光栅化器能高效将3D高斯点阵投影到2D相机视图，本文聚焦于将2D图像信息（如颜色、神经特征或分割掩码）高效反向映射至已有高斯点阵场景。这种逆向操作支持场景重光照、风格迁移与3D语义分割等应用，但也带来视角依赖着色和遮挡处理等挑战。本文方法利用法向方程求解每个高斯点的可见性加权最小二乘问题，可借助现有可微光栅化器高效实现。我们在场景重光照、特征丰富化与3D语义分割任务上验证了方法有效性，相比基于梯度下降的基线方法提速最高达一个数量级。
原文摘要 · Abstract (English)
Gaussian Splatting has recently become one of the most popular frameworks for photorealistic 3D scene reconstruction and rendering. While current rasterizers allow for efficient mappings of 3D Gaussian splats onto 2D camera views, this work focuses on mapping 2D image information (e.g. color, neural features or segmentation masks) efficiently back onto an existing scene of Gaussian splats. This 'opposite' direction enables applications ranging from scene relighting and stylization to 3D semantic segmentation, but also introduces challenges, such as view-dependent colorization and occlusion handling. Our approach tackles these challenges using the normal equation to solve a visibility-weighted least squares problem for every Gaussian and can be implemented efficiently with existing differentiable rasterizers. We demonstrate the effectiveness of our approach on scene relighting, feature enrichment and 3D semantic segmentation tasks, achieving up to an order of magnitude speedup compared to gradient descent-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。