通过显式3D眼球结构实现逼真眼神重定向，提升图像质量与准确率。
Roll Your Eyes: Gaze Redirection via Explicit 3D Eyeball Rotation
- 用3D高斯泼溅构建显式眼球结构，直接旋转平移控制眼神
- 在ETH-XGaze数据集上生成效果更真实，眼神精度优于现有方法
- 新增自适应变形模块，可模拟眼周细微肌肉运动
我们提出一种新型3D眼神重定向框架，采用显式3D眼球结构。现有方法多基于神经辐射场（NeRF），依赖体积渲染的隐式神经表示，其3D结构的旋转与平移未被显式建模。相较之下，本方法使用3D高斯泼溅（3DGS）构建专用3D眼球结构，通过显式旋转和平移实现目标眼神方向的精准生成。此外，我们设计了自适应变形模块，以复现眼周细微肌肉动作。在ETH-XGaze数据集上的实验表明，该框架能生成多样化的新型眼神图像，在图像质量与眼神估计准确率上均超越当前最优方法。
原文摘要 · Abstract (English)
We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure. Existing gaze redirection methods are typically based on neural radiance fields, which employ implicit neural representations via volume rendering. Unlike these NeRF-based approaches, where the rotation and translation of 3D representations are not explicitly modeled, we introduce a dedicated 3D eyeball structure to represent the eyeballs with 3D Gaussian Splatting (3DGS). Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure. In addition, we propose an adaptive deformation module that enables the replication of subtle muscle movements around the eyes. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our framework is capable of generating diverse novel gaze images, achieving superior image quality and gaze estimation accuracy compared to previous state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。