让大模型自主编写并调试代码来检测图像伪造,提升分析灵活性与可解释性。
Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection
- 大模型自动生成、执行并迭代优化Python工具进行低层特征分析。
- 在10万张图像上验证,能有效识别复杂伪造手法,推理能力显著提升。
- 适合需要高可解释性伪造检测的AI安全、媒体可信度研究者。
现有图像伪造检测(IFD)方法或依赖低层、语义无关的痕迹,或依赖具备高层语义知识的多模态大语言模型(MLLM)。尽管两者天然互补,但其范式与推理方式差异巨大,难以统一建模跨层次交互。为此,我们提出ForenAgent,一种多轮交互式IFD框架,使MLLM能够自主生成、执行并迭代优化围绕检测目标的Python低层工具,实现更灵活、可解释的伪造分析。ForenAgent采用冷启动与强化微调相结合的两阶段训练流程,逐步增强工具交互能力与推理适应性。受人类推理启发,设计包含全局感知、局部聚焦、迭代探测与整体判断的动态推理循环,并将其作为数据采样策略与任务对齐的奖励机制。为系统化训练与评估,构建FABench数据集,包含10万张图像及约20万条代理交互问答对。实验表明,当辅以低层工具时,ForenAgent在挑战性IFD任务中展现出涌现的工具使用能力和反思性推理,为通用型伪造检测开辟了新路径。代码将在审稿结束后发布。
原文摘要 · Abstract (English)
Existing image forgery detection (IFD) methods either exploit low-level, semantics-agnostic artifacts or rely on multimodal large language models (MLLMs) with high-level semantic knowledge. Although naturally complementary, these two information streams are highly heterogeneous in both paradigm and reasoning, making it difficult for existing methods to unify them or effectively model their cross-level interactions. To address this gap, we propose ForenAgent, a multi-round interactive IFD framework that enables MLLMs to autonomously generate, execute, and iteratively refine Python-based low-level tools around the detection objective, thereby achieving more flexible and interpretable forgery analysis. ForenAgent follows a two-stage training pipeline combining Cold Start and Reinforcement Fine-Tuning to enhance its tool interaction capability and reasoning adaptability progressively. Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication, and instantiate it as both a data-sampling strategy and a task-aligned process reward. For systematic training and evaluation, we construct FABench, a heterogeneous, high-quality agent-forensics dataset comprising 100k images and approximately 200k agent-interaction question-answer pairs. Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks when assisted by low-level tools, charting a promising route toward general-purpose IFD. The code will be released after the review process is completed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。