用深度学习自动稳定人脸网格,分离表情与头部运动。
Learning to Stabilize Faces
- 将稳定问题建模为回归任务,直接预测两帧间刚性变换。
- 在离散表情和动态表演上均优于现有方法,误差更低。
- 完全自动,无需人工干预,适合影视游戏等高精度需求场景。
如今,人脸扫描可实现高质量自动注册,但生成的网格常需进一步处理:需稳定以消除头部非必要运动。稳定对游戏开发、电影制作等任务至关重要,能将面部表情与刚性头部运动清晰分离。传统方法仍依赖人工输入、对齐不准、依赖可疑启发式或优化缓慢,且要求时间有序输入。为此,我们提出一种新型基于学习的方法,简单且完全自动。将稳定视为回归问题:给定两帧人脸网格,网络直接预测将其颅骨对齐的刚性变换。通过3D形态模型(3DMM)生成合成训练数据,利用3DMM参数可分离颅骨运动与面部皮肤运动的特性。大量实验表明,该方法在离散表情与动态表演的稳定任务中,定量与定性表现均超越当前最优水平。我们还提供了消融研究,阐明设计选择与最佳实践,便于他人应用。补充视频见项目主页 syntec-research.github.io/FaceStab。
原文摘要 · Abstract (English)
Nowadays, it is possible to scan faces and automatically register them with high quality. However, the resulting face meshes often need further processing: we need to stabilize them to remove unwanted head movement. Stabilization is important for tasks like game development or movie making which require facial expressions to be cleanly separated from rigid head motion. Since manual stabilization is labor-intensive, there have been attempts to automate it. However, previous methods remain impractical: they either still require some manual input, produce imprecise alignments, rely on dubious heuristics and slow optimization, or assume a temporally ordered input. Instead, we present a new learning-based approach that is simple and fully automatic. We treat stabilization as a regression problem: given two face meshes, our network directly predicts the rigid transform between them that brings their skulls into alignment. We generate synthetic training data using a 3D Morphable Model (3DMM), exploiting the fact that 3DMM parameters separate skull motion from facial skin motion. Through extensive experiments we show that our approach outperforms the state-of-the-art both quantitatively and qualitatively on the tasks of stabilizing discrete sets of facial expressions as well as dynamic facial performances. Furthermore, we provide an ablation study detailing the design choices and best practices to help others adopt our approach for their own uses. Supplementary videos can be found on the project webpage syntec-research.github.io/FaceStab.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。