提出公平生成人脸非语言行为的新模型，降低性别偏差影响。
Mitigation of gender bias in automatic facial non-verbal behaviors generation
- 引入性别判别器与梯度反向层，实现生成过程中的性别敏感度抑制。
- 实验显示生成行为中性别特征被显著削弱，判别器准确率大幅下降。
- 适合关注伦理安全的社交智能体开发者和人机交互研究者。
针对社交交互智能体的非语言行为生成研究，主要关注行为的可信度与与语音的同步性。然而，现有深度学习模型常继承训练数据中的偏见，引发应用中的伦理问题。本文首先分析性别对面部非语言行为（注视、头部动作、面部表情）的影响，构建一个可从非语言线索中识别说话人性别的分类器，在真实行为数据（使用先进工具提取）与先前工作生成的合成数据上均取得高准确率。在此基础上，提出新模型 FairGenderGen，将性别判别器与梯度反转层嵌入原有生成模型，从语音特征生成非语言行为，有效降低生成行为中的性别敏感性。实验表明，初始阶段训练的分类器已无法有效区分生成行为对应的说话人性别。
原文摘要 · Abstract (English)
Research on non-verbal behavior generation for social interactive agents focuses mainly on the believability and synchronization of non-verbal cues with speech. However, existing models, predominantly based on deep learning architectures, often perpetuate biases inherent in the training data. This raises ethical concerns, depending on the intended application of these agents. This paper addresses these issues by first examining the influence of gender on facial non-verbal behaviors. We concentrate on gaze, head movements, and facial expressions. We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. This classifier achieves high accuracy on both real behavior data, extracted using state-of-the-art tools, and synthetic data, generated from a model developed in previous work.Building upon this work, we present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model. This new model generates facial non-verbal behaviors from speech features, mitigating gender sensitivity in the generated behaviors. Our experiments demonstrate that the classifier, developed in the initial phase, is no longer effective in distinguishing the gender of the speaker from the generated non-verbal behaviors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。