用3D高斯点云实现高保真眼神重定向，速度更快、细节更丰富。
GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting
- 分脸与眼区域的双流3D高斯模型，独立控制眼神方向。
- 在多个数据集上达到更高眼神重定向准确率和渲染速度。
- 支持跨人物泛化，适合虚拟人、影视特效等场景使用。
gaze估计在分布外数据上面临泛化挑战。现有方法采用神经辐射场（NeRF）生成增强数据，但计算成本高且缺乏面部细节。3D高斯点云（3DGS）已成为神经场主流表示方式。尽管3DGS已广泛应用于头部虚拟形象，但在精确眼神控制和跨主体泛化方面仍存不足。本文提出GazeGaussian，首个基于双流3DGS的高保真眼神重定向方法，分别建模人脸与眼部区域。利用3DGS无结构特性，设计基于目标注视方向的刚性眼球旋转表示。为实现跨主体泛化，引入表达引导模块，将个体特异性信息注入神经渲染器。大量实验表明，GazeGaussian在渲染速度、眼神重定向精度及面部合成效果上均优于现有方法，在多个数据集上表现优异。代码已公开：https://ucwxb.github.io/GazeGaussian。
原文摘要 · Abstract (English)
Gaze estimation encounters generalization challenges when dealing with out-of-distribution data. To address this problem, recent methods use neural radiance fields (NeRF) to generate augmented data. However, existing methods based on NeRF are computationally expensive and lack facial details. 3D Gaussian Splatting (3DGS) has become the prevailing representation of neural fields. While 3DGS has been extensively examined in head avatars, it faces challenges with accurate gaze control and generalization across different subjects. In this work, we propose GazeGaussian, the first high-fidelity gaze redirection method that uses a two-stream 3DGS model to represent the face and eye regions separately. Leveraging the unstructured nature of 3DGS, we develop a novel representation of the eye for rigid eye rotation based on the target gaze direction. To enable synthesis generalization across various subjects, we integrate an expression-guided module to inject subject-specific information into the neural renderer. Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets. The code is available at: https://ucwxb.github.io/GazeGaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。