用生成模型预训练人脸美学预测,效果远超传统方法。
Generative Pre-training for Subjective Tasks: A Diffusion Transformer-Based Framework for Facial Beauty Prediction
- 用扩散Transformer在无标签人脸数据上自监督训练,学习面部本质分布。
- 在FBP5500数据集上达到0.932的皮尔逊相关系数,刷新纪录。
- 适合需要高精度主观视觉评估的研究者和应用开发者。
人脸美学预测(FBP)因主观性强及影响感知的细微整体特征而具挑战性。现有方法多基于通用图像分类预训练的卷积网络或Vision Transformer,难以捕捉与审美判断对齐的特征。本文提出两阶段框架:首先在大规模无标签人脸数据集FFHQ上,通过自监督去噪任务预训练一个扩散Transformer,使其学习人脸基础数据分布,捕获美学评估所需的关键细节与结构先验;其次,将预训练且冻结的编码器作为主干特征提取器,仅微调轻量回归头于目标数据集FBP5500。所提方法Diff-FBP在FBP5500基准上达到0.932的皮尔逊相关系数(PCC),显著超越先前基于通用预训练的方法。大量消融实验验证,生成式预训练是性能跃升的关键,生成的特征表示对主观视觉任务更具语义表征力。
原文摘要 · Abstract (English)
Facial Beauty Prediction (FBP) is a challenging computer vision task due to its subjective nature and the subtle, holistic features that influence human perception. Prevailing methods, often based on deep convolutional networks or standard Vision Transformers pre-trained on generic object classification (e.g., ImageNet), struggle to learn feature representations that are truly aligned with high-level aesthetic assessment. In this paper, we propose a novel two-stage framework that leverages the power of generative models to create a superior, domain-specific feature extractor. In the first stage, we pre-train a Diffusion Transformer on a large-scale, unlabeled facial dataset (FFHQ) through a self-supervised denoising task. This process forces the model to learn the fundamental data distribution of human faces, capturing nuanced details and structural priors essential for aesthetic evaluation. In the second stage, the pre-trained and frozen encoder of our Diffusion Transformer is used as a backbone feature extractor, with only a lightweight regression head being fine-tuned on the target FBP dataset (FBP5500). Our method, termed Diff-FBP, sets a new state-of-the-art on the FBP5500 benchmark, achieving a Pearson Correlation Coefficient (PCC) of 0.932, significantly outperforming prior art based on general-purpose pre-training. Extensive ablation studies validate that our generative pre-training strategy is the key contributor to this performance leap, creating feature representations that are more semantically potent for subjective visual tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。