arXiv:2412.11762cs.CVcs.GR2024-12被引 8

用2D高斯表示场景,实现高效无视角限制的投影映射。

GS-ProCams: Gaussian Splatting-based Projector-Camera Systems

  • 用2D高斯建模投影表面几何与材质,结合物理渲染联合优化映射关系。
  • 相比顶尖NeRF方法,训练显存降低10倍,推理速度提升900倍。
  • 无需额外光源设备,适合实时投影应用与复杂表面映射。

我们提出GS-ProCams,首个基于高斯点阵的投影-相机系统框架。该方法不仅支持无视角限制的投影映射(ProCams),还显著提升其效率。传统CNN-based ProCams受限于特定视角,而NeRF-based方法虽支持无视角映射,但需额外共置光源且计算与内存开销大。GS-ProCams采用2D高斯表示场景,显式建模投影响应、表面几何与材质、以及全局光照,通过可微分物理渲染从多视角投影中联合估计这些映射。相比最先进的NeRF方法,本方法无需额外设备,模拟质量更优;训练仅需1/10 GPU内存,推理速度提升900倍。项目页面提供代码与数据集:https://realqingyue.github.io/GS-ProCams/

原文摘要 · Abstract (English)

We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams is not only view-agnostic but also significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional co-located light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the projection surface's geometry and materials represented by Gaussians, and the global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It also uses only 1/10 of the GPU memory for training and is 900 times faster in inference speed. Please refer to our project page for the code and dataset: https://realqingyue.github.io/GS-ProCams/.

投影映射高斯点阵物理渲染实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。