首个支持表情/角度变化的面部身份定制数据集，让换脸更自然。
Turn That Frown Upside Down: FaceID Customization via Cross-Training Data
- 构建跨姿态表情的面部图像对，实现可控人脸修改。
- 40,000组图文对，每人人均20张不同特征图像。
- 提升模型在保持身份一致下的表情/视角变换能力，适合应用开发。
现有面部身份（FaceID）定制方法表现良好，但仅限生成与输入完全相同的面孔。而在真实场景中，用户常希望生成同一人不同表情（如微笑、愤怒）或角度（如侧脸）的图像。该局限源于缺乏具备受控输入输出面部变化的数据集，限制了模型学习有效修改的能力。为此，我们提出CrossFaceID，首个大规模、高质量且公开可用的专用数据集，旨在提升FaceID定制模型的面部修改能力。CrossFaceID包含约40,000个文本-图像对，覆盖约2,000人，每人平均20张图像，涵盖姿态、表情、角度和装饰等多样化面部属性。训练时以特定人脸为输入，强制模型生成同一人的不同特征图像，从而在推理阶段获得个性化修改能力。实验表明，基于CrossFaceID微调的模型，在保持身份保真度的同时，显著提升定制化能力。为推动领域发展，代码、数据集及训练模型均已公开。
原文摘要 · Abstract (English)
Existing face identity (FaceID) customization methods perform well but are limited to generating identical faces as the input, while in real-world applications, users often desire images of the same person but with variations, such as different expressions (e.g., smiling, angry) or angles (e.g., side profile). This limitation arises from the lack of datasets with controlled input-output facial variations, restricting models' ability to learn effective modifications. To address this issue, we propose CrossFaceID, the first large-scale, high-quality, and publicly available dataset specifically designed to improve the facial modification capabilities of FaceID customization models. Specifically, CrossFaceID consists of 40,000 text-image pairs from approximately 2,000 persons, with each person represented by around 20 images showcasing diverse facial attributes such as poses, expressions, angles, and adornments. During the training stage, a specific face of a person is used as input, and the FaceID customization model is forced to generate another image of the same person but with altered facial features. This allows the FaceID customization model to acquire the ability to personalize and modify known facial features during the inference stage. Experiments show that models fine-tuned on the CrossFaceID dataset retain its performance in preserving FaceID fidelity while significantly improving its face customization capabilities. To facilitate further advancements in the FaceID customization field, our code, constructed datasets, and trained models are fully available to the public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。