arXiv:2608.01661cs.CV2026-08

用视觉语言模型提升深度伪造检测的公平性与泛化能力

FairForensics: Seeing Expressions and Parsing Demographics via Vision-Language Modeling for Generalizable Fair Deepfake Detection

论文配图:FairForensics: Seeing Expressions and Parsing Demographics via Vision-Language Modeling for Generalizable Fair Deepfake Detection
图 1 · 摘自论文原文
  • 构建平衡人口分布的数据集,改进检测模型在不同群体间的公平性
  • 通过表情感知编码器和身份干扰模块,减少伪造特征中的身份偏差
  • 引入语言对齐机制,使模型更关注伪造痕迹而非性别、肤色等特征

公平性深度伪造检测面临挑战。现有方法在未见伪造类型和不同人口群体间泛化能力差,且常在人口失衡数据上训练,导致对少数群体预测偏倚。本文构建了一个新的、人口均衡的深度伪造检测基准,用于在平衡与非平衡分布下训练和评估模型公平性。提出一种名为FairForensics的新型表达与人口感知视觉-语言模型,实现伪造泛化增强和人口公平性正则化。在伪造泛化增强中,基于真实与伪造表情向量分布差异的发现,设计了表达编码器捕捉高层次表达引导的伪造模式,并引入表达感知视觉编码器融合全局外观与表情伪造特征,同时通过身份感知补丁扰动模块缓解身份偏差。在人口公平性正则化中,提出人口引导的语言编码器提取群体感知的全局语言嵌入,通过视觉-语言对齐增强伪造特征与人口信息的解耦。设计了群体感知原型公平目标,强制不同群体间类间分离与类内对齐。在新基准上的大量实验表明,该方法在泛化性和公平性上均达到当前最优。

原文摘要 · Abstract (English)

The challenge of fair deepfake detection (FDD) has attracted increasing attention. Existing fairness-enhanced detectors often suffer from suboptimal generalization to unseen manipulations and fairness across demographic groups. They are typically developed and evaluated on demographically imbalanced distributions, resulting in biased predictions toward minority groups. In this paper, we construct a novel demographically balanced FDD benchmark to train and evaluate the fairness of detectors under both balanced and imbalanced population scenarios. Additionally, we introduce a novel expression and demographic perceptual vision-language model, termed FairForensics, for generalizable fair deepfake detection. FairForensics conducts face forgery generalization enhancement and demographic-aware fairness regularization. During face forgery generalization enhancement, built upon the novel observation of significant distribution differences between pristine and forged expression vectors, we design an expression encoder to capture high-level expression-guided forgery patterns, and an expression-perceptual visual encoder that integrates global appearance and expression forgery features while mitigating identity bias using an identity-aware patch perturbation module. Under demographic-aware fairness regularization, we propose a demographic-guided language encoder to extract population-aware global language embeddings, which boosts the decoupling of forgery features from demographic information via vision-language alignment. We devise a population-aware prototype fairness objective to enforce both inter-class separability and intra-class alignment across demographic subgroups. Extensive experiments on our balanced demographic benchmark show that our method achieves the state-of-the-art in terms of generalization and fairness.

深度伪造检测公平性视觉语言模型人脸伪造

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。