让AI能听懂指令，实时分析深伪视频
Towards Interactive Deepfake Analysis
- 用GPT辅助构建指令数据集，训练可理解人类指令的多模态模型
- 推出DFA-Bench评测基准，覆盖检测、分类与特征描述三类任务
- 实现交互式分析系统DFA-GPT，支持低资源微调，适合研究者使用
现有深度伪造分析方法主要依赖判别式模型，应用场景受限。本文探索通过指令微调多模态大语言模型（MLLMs）实现交互式深伪分析，面临数据集匮乏和训练效率低等挑战。为此，我们提出：(1) 基于GPT辅助的数据构建流程，生成名为DFA-Instruct的指令跟随数据集；(2) 设计DFA-Bench评测基准，全面评估MLLM在深伪检测、分类及伪影描述方面的能力；(3) 构建交互式深伪分析系统DFA-GPT，采用低秩适配（LoRA）模块作为社区强基线。相关数据集与代码将公开于https://github.com/lxq1000/DFA-Instruct，以推动后续研究。
原文摘要 · Abstract (English)
Existing deepfake analysis methods are primarily based on discriminative models, which significantly limit their application scenarios. This paper aims to explore interactive deepfake analysis by performing instruction tuning on multi-modal large language models (MLLMs). This will face challenges such as the lack of datasets and benchmarks, and low training efficiency. To address these issues, we introduce (1) a GPT-assisted data construction process resulting in an instruction-following dataset called DFA-Instruct, (2) a benchmark named DFA-Bench, designed to comprehensively evaluate the capabilities of MLLMs in deepfake detection, deepfake classification, and artifact description, and (3) construct an interactive deepfake analysis system called DFA-GPT, as a strong baseline for the community, with the Low-Rank Adaptation (LoRA) module. The dataset and code will be made available at https://github.com/lxq1000/DFA-Instruct to facilitate further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。