通过多视角不一致检测伪造,提升模型在分布外场景的可信度。
Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning

- 融合视觉、语义与结构三路证据,捕捉伪造线索
- 跨数据集测试下准确率领先,且预测更校准
- 适合高可靠性要求的反伪造系统部署
安全关键的生物识别与取证应用需要精确预测和可靠的置信度估计,尤其在分布外情况下。深度伪造检测面临基础模型过度自信的问题,限制了实际部署。本文提出一种不确定性感知的深度伪造检测框架,通过多视角互补证据间的不一致性识别伪造。该框架包含三路:基于改进CLIP的视觉流、通过可微约束建模面部属性一致性的语义流,以及捕捉语义与取证特征间类别依赖模式的结构流。为有效融合信号,引入跨分支分歧校准(IBDC)机制,将预测不确定性与各证据流之间的分歧关联。在以FaceForensics++为训练源的跨数据集实验中,该框架在多个分布外基准上达到最先进性能,同时显著提升校准性和选择性预测表现。结果表明,结合互补证据与分歧感知的不确定性建模,可在分布偏移下构建更可信、更校准的深度伪造检测体系。
原文摘要 · Abstract (English)
Security-critical biometric and forensic applications require accurate predictions and reliable confidence estimates, particularly under distribution shift. This challenge is especially acute for deepfake detection, where foundation-model-based detectors often exhibit overconfident predictions on out-of-distribution manipulations, which limits their suitability for operational deployment. We propose an uncertainty-aware deepfake detection framework that identifies manipulations through inconsistencies across complementary evidence sources. The framework integrates three streams: a visual stream based on an adapted CLIP encoder, a semantic stream that models consistency among facial attributes through differentiable constraints, and a structural stream that captures class-dependent dependency patterns between semantic and forensic features. To effectively combine these signals, we introduce Inter-Branch Disagreement Calibration (IBDC), a disagreement-aware uncertainty modeling mechanism that links predictive uncertainty to conflicts among evidence streams. Extensive cross-dataset experiments using FaceForensics++ as the training source demonstrate that the proposed framework achieves state-of-the-art generalization across multiple out-of-distribution benchmarks while consistently improving calibration and selective prediction performance. These results show that combining complementary evidence with disagreement-aware uncertainty provides a robust foundation for trustworthy and well-calibrated deepfake detection under distribution shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。