通过追踪视频对话中眼神焦点检测深度伪造，准确率达82%。
DeepFake Detection in Dyadic Video Calls using Point of Gaze Tracking
- 利用对话时的视线焦点作为生物特征，识别非自然眼神行为。
- 在自建数据集上实现82%的检测准确率，首次应用视线追踪于实时深伪检测。
- 适合安全防护、远程会议系统等场景，对恶意攻击有预警能力。
随着深度伪造技术的进步，实时生成逼真的虚假视频已成为可能。恶意行为者开始在视频会议中利用此技术进行实时网络钓鱼攻击。视频通话的特性使我们能够获取深度伪造对象“所见”内容，即其屏幕上显示的画面。结合从攻击者视频流中估计出的视线方向，可推断深度伪造对象的视线焦点位置。由于对话中的视线焦点并非随机，而是微妙的非语言交流信号，而深度伪造无法真实模仿这一行为，因此可用来检测异常。本文提出一种针对此类攻击的实时深度伪造检测方法，利用此前未被使用的生物特征信息。模型基于对双人对话中视线模式研究的深入分析，筛选出可解释特征。我们在自建数据集上测试该模型，达到82%的准确率。这是首个利用视线焦点追踪进行深度伪造检测的方法。
原文摘要 · Abstract (English)
With recent advancements in deepfake technology, it is now possible to generate convincing deepfakes in real-time. Unfortunately, malicious actors have started to use this new technology to perform real-time phishing attacks during video meetings. The nature of a video call allows access to what the deepfake is ``seeing,'' that is, the screen displayed to the malicious actor. Using this with the estimated gaze from the malicious actors streamed video enables us to estimate where the deepfake is looking on screen, the point of gaze. Because the point of gaze during conversations is not random and is instead used as a subtle nonverbal communicator, it can be used to detect deepfakes, which are not capable of mimicking this subtle nonverbal communication. This paper proposes a real-time deepfake detection method adapted to this genre of attack, utilizing previously unavailable biometric information. We built our model based on explainable features selected after careful review of research on gaze patterns during dyadic conversations. We then test our model on a novel dataset of our creation, achieving an accuracy of 82\%. This is the first reported method to utilize point-of-gaze tracking for deepfake detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。