用少量参数实现高精度人脸个性化生成，保留皱纹纹身等细节。
Low-Rank Head Avatar Personalization with Registers
- 设计可学习的3D特征存储模块，增强LoRA对高频细节的捕捉能力。
- 仅用少量参数即可还原未见过的人脸，生成效果优于现有方法。
- 适合需要快速定制真实人脸动画的数字人应用开发人员。
我们提出一种新的低秩方法，用于通用头像生成模型的个性化。以往工作通过大规模多身份数据集训练出高质量人脸动画模型，但难以捕捉个体特有细节，因模型学习的是通用先验。现有低秩适配（LoRA）方案在还原高频率面部细节时仍存在挑战。为此，我们设计了注册模块（Register Module），将其应用于预训练模型的中间特征，以可学习的3D特征空间存储并复用信息，显著提升适配性能，同时仅需极少参数即可适应未见身份。为验证方法有效性，我们收集了包含显著面部特征（如皱纹、纹身）的个人说话视频数据集。实验表明，该方法能忠实还原未见过的人脸，在定量与定性评估中均优于现有方法。代码、模型及数据集将公开发布。
原文摘要 · Abstract (English)
We introduce a novel method for low-rank personalization of a generic model for head avatar generation. Prior work proposes generic models that achieve high-quality face animation by leveraging large-scale datasets of multiple identities. However, such generic models usually fail to synthesize unique identity-specific details, since they learn a general domain prior. To adapt to specific subjects, we find that it is still challenging to capture high-frequency facial details via popular solutions like low-rank adaptation (LoRA). This motivates us to propose a specific architecture, a Register Module, that enhances the performance of LoRA, while requiring only a small number of parameters to adapt to an unseen identity. Our module is applied to intermediate features of a pre-trained model, storing and re-purposing information in a learnable 3D feature space. To demonstrate the efficacy of our personalization method, we collect a dataset of talking videos of individuals with distinctive facial details, such as wrinkles and tattoos. Our approach faithfully captures unseen faces, outperforming existing methods quantitatively and qualitatively. We will release the code, models, and dataset to the public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。