用三重高斯点阵实现快速高质量光照重渲染，支持真实感阴影与全局光照。
GS^3: Efficient Relighting with Triple Gaussian Splatting
- 用空间+角度高斯混合模型描述复杂材质反射特性。
- 训练仅需40-70分钟，单卡渲染达90帧每秒。
- 适合影视、游戏等需要实时重光照的场景重建应用。
我们提出一种基于空间与角度高斯的表示方法及三重点阵渲染流程，实现从多视角点光源输入图像中实时生成高质量的新视角与新光照效果。为描述复杂外观，每个空间高斯采用朗伯模型加角度高斯混合作为有效反射函数。为生成自阴影，将所有空间高斯向光源方向投射以获取阴影值，并通过小型多层感知机进行优化。为补偿全局光照等效应，另训练一个网络为每个空间高斯计算并添加对应的RGB元组。在30个具有广泛几何形态（从实体到蓬松）与外观特性（从半透明到各向异性）的样本上验证了该方法的有效性，涵盖合成/重建物体的渲染图、手持相机配闪光灯拍摄的照片以及专业光场扫描数据。训练时间40-70分钟，单卡渲染速度达90 fps。结果在质量与性能上均优于当前最优技术。代码与数据公开于https://GSrelight.github.io/。
原文摘要 · Abstract (English)
We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex appearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance. Our code and data are publicly available at https://GSrelight.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。