构建社交平台深伪检测数据集与多模态识别模型
SocialDF: Benchmark Dataset and Detection Model for Mitigating Harmful Deepfake Content on Social Media Platforms
- 用大模型融合人脸、语音、语义多维度验证
- 在真实社交生态中采集高保真深伪样本
- 适合安全团队和平台方部署防伪系统
深度生成模型的快速发展显著提升了合成媒体的真实感，带来机遇的同时也引发安全挑战。尽管深伪技术在娱乐和无障碍领域有应用价值，但已成为社交媒体上误导性信息传播的重要手段。现有检测框架难以区分良性与蓄意操纵公众认知的对抗性深伪内容。为此，我们提出SocialDF，一个反映社交媒体真实深伪挑战的精选数据集，涵盖来自多种在线生态的高保真深伪样本，全面覆盖各类操纵手法。我们设计了一种基于大模型的多因素检测方法，结合人脸识别、自动语音转录及多智能体大模型流水线，交叉验证音视频线索。该方法强调鲁棒的多模态验证，融入语言、行为与上下文分析，有效区分合成内容与真实内容。
原文摘要 · Abstract (English)
The rapid advancement of deep generative models has significantly improved the realism of synthetic media, presenting both opportunities and security challenges. While deepfake technology has valuable applications in entertainment and accessibility, it has emerged as a potent vector for misinformation campaigns, particularly on social media. Existing detection frameworks struggle to distinguish between benign and adversarially generated deepfakes engineered to manipulate public perception. To address this challenge, we introduce SocialDF, a curated dataset reflecting real-world deepfake challenges on social media platforms. This dataset encompasses high-fidelity deepfakes sourced from various online ecosystems, ensuring broad coverage of manipulative techniques. We propose a novel LLM-based multi-factor detection approach that combines facial recognition, automated speech transcription, and a multi-agent LLM pipeline to cross-verify audio-visual cues. Our methodology emphasizes robust, multi-modal verification techniques that incorporate linguistic, behavioral, and contextual analysis to effectively discern synthetic media from authentic content.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。