仅用一张照片生成可连续编辑面部特征的3D虚拟人
PERSE: Personalized 3D Generative Avatars from A Single Portrait
- 用合成2D视频数据训练，实现面部属性连续变化
- 在保持身份一致的前提下，支持精细表情与姿态控制
- 适合需要个性化虚拟形象的数字人、游戏开发场景
我们提出PERSE，一种从单张肖像图构建个性化3D生成虚拟人的方法。该虚拟人可在连续且解耦的潜在空间中编辑面部属性，同时保持个体身份不变。方法首先合成大规模合成2D视频数据集，每个视频包含一致的表情、视角变化以及特定面部属性的连续变化。我们设计了一种新管道，生成高质量、逼真的带属性编辑的2D视频。基于此合成数据集，我们提出一种基于3D高斯泼溅的个性化虚拟人生成方法，学习可直观操作的连续解耦潜在空间。为确保潜在空间平滑过渡，引入以插值2D人脸为监督的潜在空间正则化技术。相比以往方法，PERSE在保持参考个体身份的同时，生成高质量可插值属性的虚拟人。
原文摘要 · Abstract (English)
We present PERSE, a method for building a personalized 3D generative avatar from a reference portrait. Our avatar enables facial attribute editing in a continuous and disentangled latent space to control each facial attribute, while preserving the individual's identity. To achieve this, our method begins by synthesizing large-scale synthetic 2D video datasets, where each video contains consistent changes in facial expression and viewpoint, along with variations in a specific facial attribute from the original input. We propose a novel pipeline to produce high-quality, photorealistic 2D videos with facial attribute editing. Leveraging this synthetic attribute dataset, we present a personalized avatar creation method based on 3D Gaussian Splatting, learning a continuous and disentangled latent space for intuitive facial attribute manipulation. To enforce smooth transitions in this latent space, we introduce a latent space regularization technique by using interpolated 2D faces as supervision. Compared to previous approaches, we demonstrate that PERSE generates high-quality avatars with interpolated attributes while preserving the identity of the reference individual.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。