无需标注数据,实现面部表情与动作的精准分离与自由操控
Motion Manipulation via Unsupervised Keypoint Positioning in Face Animation
- 通过自监督学习在隐空间解耦表情与运动信息
- 新关键点计算方法支持任意动作控制,生成更自然动画
- 首次在无监督框架下实现表情连续插值,适合影视特效开发
面部动画广泛应用于虚拟形象生成。现有基于无监督关键点定位的方法虽能生成逼真细节,但难以实现可控生成,因关键点分解流程无法完全分离身份语义与纠缠的运动信息(如旋转、平移、表情)。为此,我们提出一种新方法——基于无监督关键点定位的面部动画运动操控(MMFA)。首先引入自监督表示学习,在隐空间编码并解码表情,使其与其它运动信息解耦;其次提出新的关键点计算方式,实现任意运动控制;此外设计变分自编码器,将表情特征映射至连续高斯分布,首次在无监督框架下实现表情插值。我们在公开数据集上进行大量实验,结果表明,相较于以往方法,MMFA在生成真实感动画和操控面部运动方面具有显著优势。
原文摘要 · Abstract (English)
Face animation deals with controlling and generating facial features with a wide range of applications. The methods based on unsupervised keypoint positioning can produce realistic and detailed virtual portraits. However, they cannot achieve controllable face generation since the existing keypoint decomposition pipelines fail to fully decouple identity semantics and intertwined motion information (e.g., rotation, translation, and expression). To address these issues, we present a new method, Motion Manipulation via unsupervised keypoint positioning in Face Animation (MMFA). We first introduce self-supervised representation learning to encode and decode expressions in the latent feature space and decouple them from other motion information. Secondly, we propose a new way to compute keypoints aiming to achieve arbitrary motion control. Moreover, we design a variational autoencoder to map expression features to a continuous Gaussian distribution, allowing us for the first time to interpolate facial expressions in an unsupervised framework. We have conducted extensive experiments on publicly available datasets to validate the effectiveness of MMFA, which show that MMFA offers pronounced advantages over prior arts in creating realistic animation and manipulating face motion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。