实时生成多元性别与族裔特征的3D人脸,让虚拟形象更包容。
Reflections on Diversity: A Real-time Virtual Mirror for Inclusive 3D Face Transformations
- 结合GAN纹理与3DMM几何建模,实现实时3D人脸变换。
- 支持非洲、亚洲、欧洲等多族裔特征映射,生成真实感强的多样性人脸。
- 可生成多人平均脸,适合用于包容性设计与人机交互研究。
实时3D人脸操控在虚拟现实、社交媒体和人机交互中具有重要应用。本文提出名为「多样性之镜」(Mirror of Diversity, MOD)的新系统,融合生成对抗网络(GANs)进行纹理变换与3D可变形模型(3DMMs)进行面部几何建模,实现反映多种人口统计特征的逼真人脸转换,强调多样性之美与人类特征的普适性。当参与者面对上方摄像头的显示器时,其面部特征被实时捕捉,并可通过转换模拟不同性别与族裔特征(如非洲、亚洲、欧洲人群)。系统另一功能「集体面孔」(Collective Face)从多人面部数据中生成平均脸表示。通过全面评估协议验证转换的真实性和人口统计准确性。定性反馈来自问卷调查,对比MOD转换与Snapchat、TikTok等平台滤镜效果;定量分析则采用预训练卷积神经网络预测性别与族裔,验证转换准确性。
原文摘要 · Abstract (English)
Real-time 3D face manipulation has significant applications in virtual reality, social media and human-computer interaction. This paper introduces a novel system, which we call Mirror of Diversity (MOD), that combines Generative Adversarial Networks (GANs) for texture manipulation and 3D Morphable Models (3DMMs) for facial geometry to achieve realistic face transformations that reflect various demographic characteristics, emphasizing the beauty of diversity and the universality of human features. As participants sit in front of a computer monitor with a camera positioned above, their facial characteristics are captured in real time and can further alter their digital face reconstruction with transformations reflecting different demographic characteristics, such as gender and ethnicity (e.g., a person from Africa, Asia, Europe). Another feature of our system, which we call Collective Face, generates an averaged face representation from multiple participants' facial data. A comprehensive evaluation protocol is implemented to assess the realism and demographic accuracy of the transformations. Qualitative feedback is gathered through participant questionnaires, which include comparisons of MOD transformations with similar filters on platforms like Snapchat and TikTok. Additionally, quantitative analysis is conducted using a pretrained Convolutional Neural Network that predicts gender and ethnicity, to validate the accuracy of demographic transformations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。