arXiv:2606.00098cs.CVeess.IV2026-06

通过选择关键面部区域提升伪造检测的泛化性与可解释性

Segmentation-Guided Spatial Indexing for Generalizable and Explainable Deepfake Detection

论文配图:Segmentation-Guided Spatial Indexing for Generalizable and Explainable Deepfake Detection
图 1 · 摘自论文原文
  • 先选有意义的面部区域,再分类,避免全脸信息干扰
  • 在Celeb-DF v2上达到0.905的AUC,优于现有方法8.1至16.9个百分点
  • 决策基于实际区域特征,无需热力图即可说明判断依据

本文提出分割引导的空间索引方法,用于通用且可解释的深度伪造检测。核心思路是反转传统流程:不先聚合全部面部令牌再分类,而是先筛选具有语义意义的图像块令牌,仅对这些保留区域进行池化。使用冻结的FaRL解析器为DINOv3 ViT-L/16的每个图像块分配语义标签,非目标块被丢弃,线性探测器对保留区域进行分类。该空间索引利用DINOv3的块级空间一致性——正是这一特性使模型具备涌现分割能力——向探测器提供更纯净的区域子空间,减少整体面部线索对篡改证据的稀释。区域归因具有结构性:当嘴部模型判定为伪造时,仅依赖嘴部令牌,而非叠加显著性图。在Celeb-DF v2数据集上,嘴部索引探测器获得0.905的AUC,超越LipForensics(+8.1 pp)和Xception(+16.9 pp),且未微调DINOv3或FaRL,也无需目标域数据。消融实验表明:若用CLS令牌替代区域选择,AUC下降26.4 pp;若用FaRL特征替代DINOv3,则下降20.9 pp。DINOv3表征与空间索引均独立必要,单独使用均无法达到完整系统性能。

原文摘要 · Abstract (English)

We introduce segmentation-guided spatial indexing for generalizable and explainable deepfake detection. The key idea reverses the standard design order: rather than pooling all facial tokens and classifying afterward, we first select semantically meaningful patch tokens, then pool only those. A frozen FaRL parser assigns each DINOv3 ViT-L/16 patch token a semantic label; non-target tokens are discarded; a linear probe classifies the retained region. This spatial indexing exploits DINOv3's patch-level spatial consistency, the same property that enables emergent segmentation, to present the probe with a purer regional subspace where manipulation-relevant evidence is less diluted by whole-face cues. Region attribution is structural: when the mouth model predicts fake, the decision used only mouth tokens, not an overlaid saliency map. On Celeb-DF v2, the mouth-indexed probe achieves AUC 0.905, outperforming LipForensics (+8.1 pp) and Xception (+16.9 pp), with no DINOv3 or FaRL fine-tuning and no target-domain data. Ablations isolate the mechanism: replacing regional selection with DINOv3's CLS token drops Celeb-DF v2 AUC by 26.4 pp; replacing DINOv3 with FaRL features drops it by 20.9 pp. Both DINOv3 representation and the spatial index are independently necessary; neither alone approaches the full system.

深度伪造检测可解释性空间索引视觉变换器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。