首个阿拉伯语-英语混用音视频深伪数据集，助力多语言场景下伪造内容检测。
Tell me Habibi, is it Real or Fake?
- 构建融合阿拉伯语与英语混用的音视频生成流水线，支持跨语言深伪检测研究。
- 包含38.7万段视频和超过765小时真实与伪造音频视频数据。
- 适用于多语言深伪检测、跨文化媒体安全研究及模型鲁棒性评估。
深度伪造生成技术迅速发展，使虚假媒体更难被识别，引发严重社会担忧。现有深伪检测与数据集研究多聚焦单语内容，忽视了多语言及语码转换语音带来的挑战——阿拉伯语与英语在阿拉伯地区广泛混合使用，这种语言交错现象会干扰仅基于单语训练的检测模型。为此，我们提出首个大规模阿拉伯语-英语多模态音视频深伪数据集ArEnAV，涵盖句内语码转换、方言差异及单语阿拉伯语内容。该数据集由四款文本转语音（TTS）模型与两款唇形同步模型联合生成，共包含38.7万段视频和超过765小时的真实与伪造音视频数据，支持多语言多模态深伪检测的系统性分析。我们在现有单语与多语数据集上进行基准测试，并对比先进深伪检测模型与人工评估，验证其在推动深伪研究方面的潜力。数据集已公开于Hugging Face。
原文摘要 · Abstract (English)
Deepfake generation methods are evolving fast, making fake media harder to detect and raising serious societal concerns. Most deepfake detection and dataset creation research focuses on monolingual content, often overlooking the challenges of multilingual and code-switched speech, where multiple languages are mixed within the same discourse. Code-switching, especially between Arabic and English, is common in the Arab world and is widely used in digital communication. This linguistic mixing poses extra challenges for deepfake detection, as it can confuse models trained mostly on monolingual data. To address this, we introduce \textbf{ArEnAV}, the first large-scale Arabic-English audio-visual deepfake dataset featuring intra-utterance code-switching, dialectal variation, and monolingual Arabic content. It \textbf{contains 387k videos and over 765 hours of real and fake videos}. Our dataset is generated using a novel pipeline integrating four Text-To-Speech and two lip-sync models, enabling comprehensive analysis of multilingual multimodal deepfake detection. We benchmark our dataset against existing monolingual and multilingual datasets, state-of-the-art deepfake detection models, and a human evaluation, highlighting its potential to advance deepfake research. The dataset can be accessed \href{https://huggingface.co/datasets/kartik060702/ArEnAV-Full}{here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。