arXiv:2509.23859cs.CV2025-09被引 2

融合CNN与视觉变换器,用对抗去偏提升人脸美丑预测的公平性与可解释性。

FairViT-GAN: A Hybrid Vision Transformer with Adversarial Debiasing for Fair and Explainable Facial Beauty Prediction

  • 双分支架构:CNN捕获局部纹理,ViT建模全局和谐
  • 实现0.9230的皮尔逊相关系数,种族间性能差距降低82.9%
  • 通过可视化展示各模块关注点,增强模型透明度

人脸美丑预测在深度学习推动下取得显著进展,但现有模型普遍存在结构限制、人口统计偏差及缺乏透明性问题。基于卷积神经网络(CNN)的方法擅长捕捉局部纹理,却难以建模全局面部和谐;而视觉变换器(ViT)虽能有效建模长程依赖,但常忽略细粒度特征。此外,训练于基准数据集的模型可能无意中习得并延续与族裔等受保护属性相关的社会偏见。为此,我们提出FairViT-GAN,一种新型混合框架,协同整合CNN分支用于局部特征提取,以及ViT分支用于全局上下文建模。更关键的是,引入对抗去偏机制,使特征提取器显式学习对受保护属性不变的表示,主动缓解算法偏见。模型透明性通过可视化各分支的关注区域得以增强。在SCUT-FBP5500基准上的大量实验表明,FairViT-GAN不仅达到新最优性能,皮尔逊相关系数达0.9230,均方根误差降至0.2650,且在公平性上表现卓越——种族子组间性能差距减少82.9%,对抗分类准确率降至近随机水平(52.1%)。我们认为FairViT-GAN为负责任的主观视觉评估AI系统提供了稳健、透明且显著公平的范本。

原文摘要 · Abstract (English)

Facial Beauty Prediction (FBP) has made significant strides with the application of deep learning, yet state-of-the-art models often exhibit critical limitations, including architectural constraints, inherent demographic biases, and a lack of transparency. Existing methods, primarily based on Convolutional Neural Networks (CNNs), excel at capturing local texture but struggle with global facial harmony, while Vision Transformers (ViTs) effectively model long-range dependencies but can miss fine-grained details. Furthermore, models trained on benchmark datasets can inadvertently learn and perpetuate societal biases related to protected attributes like ethnicity. To address these interconnected challenges, we propose \textbf{FairViT-GAN}, a novel hybrid framework that synergistically integrates a CNN branch for local feature extraction and a ViT branch for global context modeling. More significantly, we introduce an adversarial debiasing mechanism where the feature extractor is explicitly trained to produce representations that are invariant to protected attributes, thereby actively mitigating algorithmic bias. Our framework's transparency is enhanced by visualizing the distinct focus of each architectural branch. Extensive experiments on the SCUT-FBP5500 benchmark demonstrate that FairViT-GAN not only sets a new state-of-the-art in predictive accuracy, achieving a Pearson Correlation of \textbf{0.9230} and reducing RMSE to \textbf{0.2650}, but also excels in fairness. Our analysis reveals a remarkable \textbf{82.9\% reduction in the performance gap} between ethnic subgroups, with the adversary's classification accuracy dropping to near-random chance (52.1\%). We believe FairViT-GAN provides a robust, transparent, and significantly fairer blueprint for developing responsible AI systems for subjective visual assessment.

人脸美丑预测视觉变换器去偏可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。