实时高质眼球方向重定向，单图生成逼真人脸注视效果
RTGaze: Real-Time 3D-Aware Gaze Redirection from a Single Image
- 学习可控制眼动的面部表征，结合神经渲染实现眼球重定向
- 单图处理仅需0.06秒，比之前最快方法快800倍，保持3D一致性
- 适合影视制作、虚拟人交互等需实时眼神控制的场景
眼球重定向技术旨在生成具有可控眼动的逼真人脸图像。然而，现有方法常面临3D一致性差、效率低或画质不足的问题，限制了实际应用。本文提出RTGaze，一种实时且高质量的眼球重定向方法。该方法从人脸图像和注视提示中学习可控制眼动的面部表征，并通过神经渲染解码实现眼球重定向。同时，利用预训练3D肖像生成器提取面部几何先验，提升生成质量。我们在多个数据集上进行定性与定量评估，结果表明，RTGaze在效率、重定向精度和图像质量方面均达到当前最优水平。系统采用前向网络，单图处理时间约为0.06秒，比之前最先进的3D-aware方法快800倍。
原文摘要 · Abstract (English)
Gaze redirection methods aim to generate realistic human face images with controllable eye movement. However, recent methods often struggle with 3D consistency, efficiency, or quality, limiting their practical applications. In this work, we propose RTGaze, a real-time and high-quality gaze redirection method. Our approach learns a gaze-controllable facial representation from face images and gaze prompts, then decodes this representation via neural rendering for gaze redirection. Additionally, we distill face geometric priors from a pretrained 3D portrait generator to enhance generation quality. We evaluate RTGaze both qualitatively and quantitatively, demonstrating state-of-the-art performance in efficiency, redirection accuracy, and image quality across multiple datasets. Our system achieves real-time, 3D-aware gaze redirection with a feedforward network (~0.06 sec/image), making it 800x faster than the previous state-of-the-art 3D-aware methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。