arXiv:2508.07596cs.CV2025-08中稿 · publicaiton in the…被引 15

让普通用户也能看懂深伪视频检测逻辑的可解释框架。

From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users

  • 融合视觉、语义与叙事三层解释,生成直观反馈。
  • 在DF40数据集上实现高精度检测与对齐热力图的解释。
  • 适合非专家在司法、新闻等场景快速判断真假。

深度伪造技术的泛滥对数字真实性构成严峻挑战,尤其在法证、新闻和法律领域。现有检测系统虽有较高分类准确率,但多为黑箱模型,缺乏透明性且难以支持人类推理,限制了其在真实决策场景中的应用,尤其对非专家用户。本文提出DF-P2E(Deepfake: Prediction to Explanation)框架,整合视觉、语义与叙事三层次解释,使深伪检测可解释且易用。该框架包含三个模块:(1) 基于Grad-CAM的深伪分类器与显著性可视化;(2) 生成被篡改区域自然语言摘要的视觉描述模块;(3) 利用微调大语言模型生成上下文敏感、用户友好的叙事解释模块。我们在当前最多样化的深伪数据集DF40上进行了实例化与评估。实验表明,该系统在保持竞争力检测性能的同时,能生成与Grad-CAM激活区域高度一致的高质量解释。通过统一预测与解释的连贯流程,本工作为对抗性媒体环境中可信、透明AI系统的构建提供了可扩展的解决方案。

原文摘要 · Abstract (English)

The proliferation of deepfake technologies poses urgent challenges and serious risks to digital integrity, particularly within critical sectors such as forensics, journalism, and the legal system. While existing detection systems have made significant progress in classification accuracy, they typically function as black-box models, offering limited transparency and minimal support for human reasoning. This lack of interpretability hinders their usability in real-world decision-making contexts, especially for non-expert users. In this paper, we present DF-P2E (Deepfake: Prediction to Explanation), a novel multimodal framework that integrates visual, semantic, and narrative layers of explanation to make deepfake detection interpretable and accessible. The framework consists of three modular components: (1) a deepfake classifier with Grad-CAM-based saliency visualisation, (2) a visual captioning module that generates natural language summaries of manipulated regions, and (3) a narrative refinement module that uses a fine-tuned Large Language Model (LLM) to produce context-aware, user-sensitive explanations. We instantiate and evaluate the framework on the DF40 benchmark, the most diverse deepfake dataset to date. Experiments demonstrate that our system achieves competitive detection performance while providing high-quality explanations aligned with Grad-CAM activations. By unifying prediction and explanation in a coherent, human-aligned pipeline, this work offers a scalable approach to interpretable deepfake detection, advancing the broader vision of trustworthy and transparent AI systems in adversarial media environments.

深伪检测可解释AI多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。