arXiv:2410.23663cs.CVcs.MM2024-10中稿 · with IEEE Trans被引 18

通过扩散学习方向不一致性,提升通用深伪视频检测效果。

DIP: Diffusion Learning of Inconsistency Pattern for General DeepFake Detection

  • 用变换器架构捕捉视频中水平与垂直方向的时序不一致模式。
  • 在多个公开数据集上达到当前最佳性能,显著提升泛化能力。
  • 适合关注视频伪造检测、尤其是时序异常分析的研究者。

随着深度伪造生成技术的发展,保护多媒体内容完整性对深伪视频检测的重要性日益凸显。近期研究发现,时序不一致线索有助于提升检测器的泛化能力。我们观察到,伪造视频在运动信息上的时序伪影通常在水平和垂直方向表现出显著不同的不一致模式,这些模式可被用于提升检测器泛化性。本文提出一种基于变换器的不一致性模式扩散学习框架(DIP),利用方向性不一致进行深伪视频检测。具体而言,DIP首先通过时空编码器表示时空信息,随后采用方向感知注意力与不一致扩散机制的解码器,探索潜在不一致模式并联合学习内在关联。此外,引入时空不变损失(STI Loss),对比经时空增强的样本对,防止模型过拟合非关键伪造特征。在多个公开数据集上的大量实验表明,该方法能有效识别方向性伪造线索,并取得当前最优性能。

原文摘要 · Abstract (English)

With the advancement of deepfake generation techniques, the importance of deepfake detection in protecting multimedia content integrity has become increasingly obvious. Recently, temporal inconsistency clues have been explored to improve the generalizability of deepfake video detection. According to our observation, the temporal artifacts of forged videos in terms of motion information usually exhibits quite distinct inconsistency patterns along horizontal and vertical directions, which could be leveraged to improve the generalizability of detectors. In this paper, a transformer-based framework for Diffusion Learning of Inconsistency Pattern (DIP) is proposed, which exploits directional inconsistencies for deepfake video detection. Specifically, DIP begins with a spatiotemporal encoder to represent spatiotemporal information. A directional inconsistency decoder is adopted accordingly, where direction-aware attention and inconsistency diffusion are incorporated to explore potential inconsistency patterns and jointly learn the inherent relationships. In addition, the SpatioTemporal Invariant Loss (STI Loss) is introduced to contrast spatiotemporally augmented sample pairs and prevent the model from overfitting nonessential forgery artifacts. Extensive experiments on several public datasets demonstrate that our method could effectively identify directional forgery clues and achieve state-of-the-art performance.

深伪检测时序分析扩散模型视频伪造

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。