用合成数据训练发型分类模型,提升对多样发型的识别公平性与鲁棒性。
Hairmony: Fairness-aware hairstyle classification
- 基于合成数据与专家共建的发型分类体系进行训练。
- 在复杂发型上表现优于现有参数化方法,显著提升识别鲁棒性。
- 适合需要包容性与高精度发型识别的应用场景,如虚拟形象构建。
我们提出一种从单张图像中预测人物发型的方法。尽管在用户数字化和虚拟体验注册中应用日益广泛,现有方法仍受限于可捕捉发型的范围。人类头发高度多样化且缺乏通用描述或分类标准,使该任务极具挑战。多数现有方法依赖于丝状级的参数化建模,但难以表征短发、卷曲发、蓬松发及束发等类型。本文采用分类方法,以更好覆盖所需发型多样性,构建真正鲁棒且包容的系统。以往分类方法受限于标注质量差、多样性不足的数据集,影响注册系统的实用性。我们仅使用合成数据训练模型,可精确控制发型属性、发色、面部特征、姿态、环境等参数,并生成无噪声的真值标签。我们与多位领域专家合作建立新型发型分类体系,用于平衡训练数据、监督模型并直接衡量公平性。我们用该体系标注了合成训练数据和真实评估数据集,并公开发布,以支持未来方法的对比。采用预训练特征提取网络架构提升对真实数据的泛化能力,并将分类体系属性作为辅助任务以提高准确率。实验表明,本方法在复杂发型上的鲁棒性显著优于近期参数化方法。
原文摘要 · Abstract (English)
We present a method for prediction of a person's hairstyle from a single image. Despite growing use cases in user digitization and enrollment for virtual experiences, available methods are limited, particularly in the range of hairstyles they can capture. Human hair is extremely diverse and lacks any universally accepted description or categorization, making this a challenging task. Most current methods rely on parametric models of hair at a strand level. These approaches, while very promising, are not yet able to represent short, frizzy, coily hair and gathered hairstyles. We instead choose a classification approach which can represent the diversity of hairstyles required for a truly robust and inclusive system. Previous classification approaches have been restricted by poorly labeled data that lacks diversity, imposing constraints on the usefulness of any resulting enrollment system. We use only synthetic data to train our models. This allows for explicit control of diversity of hairstyle attributes, hair colors, facial appearance, poses, environments and other parameters. It also produces noise-free ground-truth labels. We introduce a novel hairstyle taxonomy developed in collaboration with a diverse group of domain experts which we use to balance our training data, supervise our model, and directly measure fairness. We annotate our synthetic training data and a real evaluation dataset using this taxonomy and release both to enable comparison of future hairstyle prediction approaches. We employ an architecture based on a pre-trained feature extraction network in order to improve generalization of our method to real data and predict taxonomy attributes as an auxiliary task to improve accuracy. Results show our method to be significantly more robust for challenging hairstyles than recent parametric approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。