用大模型自动识别生成视频,无需训练就能高效检测。
LAVID: An Agentic LVLM Framework for Diffusion-Generated Video Detection
- 让大模型调用外部工具并自动生成提示,提升检测能力。
- 在多个数据集上比顶尖方法平均提升6.2%至30.2%的准确率。
- 适合做AI内容安全、数字真实性验证的研究者和开发者。
生成模型在高质量视频创作方面取得显著进展,引发数字完整性与隐私安全担忧。尽管图像领域已有大量深度伪造检测研究,视频领域的检测仍处于空白。大型视觉语言模型(LVLM)凭借强大的推理与多模态能力,成为新型检测工具,突破了传统深度学习方法透明性差、难以识别新伪影的局限。受此启发,我们提出LAVID,一种基于LVLM的生成视频检测框架,通过显式知识增强实现检测。核心思路包括:(1) 领先的LVLM可调用外部工具获取有用信息以辅助检测;(2) 提示结构设计影响模型对视频内容的理解能力。我们的方案自动选择一组显式知识工具,并通过自我重写动态调整提示结构。不同于以往需训练额外检测器的方法,本方法完全免训练,仅依赖LVLM推理即可完成检测。为支持研究,我们构建了新基准数据集VideoFor,包含来自多种视频生成工具的高质量生成视频。评估结果显示,LAVID在四个SOTA LVLM上相较最先进基线,F1分数提升6.2%至30.2%。
原文摘要 · Abstract (English)
The impressive achievements of generative models in creating high-quality videos have raised concerns about digital integrity and privacy vulnerabilities. Recent works of AI-generated content detection have been widely studied in the image field (e.g., deepfake), yet the video field has been unexplored. Large Vision Language Model (LVLM) has become an emerging tool for AI-generated content detection for its strong reasoning and multimodal capabilities. It breaks the limitations of traditional deep learning based methods faced with like lack of transparency and inability to recognize new artifacts. Motivated by this, we propose LAVID, a novel LVLMs-based ai-generated video detection with explicit knowledge enhancement. Our insight list as follows: (1) The leading LVLMs can call external tools to extract useful information to facilitate its own video detection task; (2) Structuring the prompt can affect LVLM's reasoning ability to interpret information in video content. Our proposed pipeline automatically selects a set of explicit knowledge tools for detection, and then adaptively adjusts the structure prompt by self-rewriting. Different from prior SOTA that trains additional detectors, our method is fully training-free and only requires inference of the LVLM for detection. To facilitate our research, we also create a new benchmark \vidfor with high-quality videos generated from multiple sources of video generation tools. Evaluation results show that LAVID improves F1 scores by 6.2 to 30.2% over the top baselines on our datasets across four SOTA LVLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。