让3D人脸眼神与头部动作风格可控且自然同步
StyGazeTalk: Learning Stylized Generation of Gaze and Head Dynamics
- 融合多模态数据生成可调节风格的注视与头部运动
- 在自建高精度数据集HAGE上实现时序一致的动作表现
- 适合虚拟角色、人机交互和沉浸式内容创作
眼神与头部动作在表达性3D媒体、人机交互和沉浸式通信中至关重要。现有方法常孤立建模面部组件，缺乏生成个性化、风格感知眼神行为的机制。我们提出StyGazeTalk，一种多模态框架，可合成风格可控的同步眼神-头部动态。为支持高保真训练，我们构建了HAGE数据集，包含眼动追踪数据、音频、头部姿态及3D面部参数。实验表明，该方法生成的时间连贯、风格一致的眼神-头部动作，显著提升了3D人脸生成的真实感。
原文摘要 · Abstract (English)
Gaze and head movements play a central role in expressive 3D media, human-agent interaction, and immersive communication. Existing works often model facial components in isolation and lack mechanisms for generating personalized, style-aware gaze behaviors. We propose StyGazeTalk, a multimodal framework that synthesizes synchronized gaze-head dynamics with controllable styles. To support high-fidelity training, we construct HAGE, a high-precision multimodal dataset containing eye-tracking data, audio, head pose, and 3D facial parameters. Experiments show that our method produces temporally coherent, style-consistent gaze-head motions, enhancing realism in 3D face generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。