通过自适应推理增强,提升多模态虚假信息检测的准确性
Agentic Mixed-Source Multi-Modal Misinformation Detection with Adaptive Test-Time Scaling
- 每个模态用多个推理路径并行分析,再选最优结果
- 在两个基准上达到当前最佳零样本检测效果
- 适合需要高可靠性虚假信息检测的应用场景
视觉语言模型(VLMs)在社交平台的多模态虚假信息检测中表现优异,尤其在无标注或标注延迟的零样本场景下。然而,单一VLM难以应对更复杂的混合来源多模态虚假信息检测(M3D)任务。以带文字说明的图片为例,虚假信息可能源于不实文本、伪造图像或图文不匹配。尽管近期的智能体系统通过连接模态专用的VLM智能体可处理零样本M3D,但其性能受限于架构设计。现有方案中,每个输入样本仅允许各智能体执行一次前向推理,易受模型随机性和复杂案例中的推理错误影响,且缺乏对替代推理路径的探索,未能充分发挥VLM的推理潜力。本文提出AgentM3D,一种用于零样本M3D的多智能体框架。为增强VLM推理能力,引入自适应测试时缩放机制:每个模态专用的VLM智能体采用Best-of-N机制,并由评分智能体进行任务对齐打分。智能体按模态特定的级联链组织,以减少冗余计算并控制错误传播。为保证可扩展性,规划智能体根据样本难度动态决定最大推理路径数,自适应停止机制防止单个智能体内过度推理。在两个M3D基准上的大量实验表明,AgentM3D在零样本检测性能上优于多种基于VLM和智能体的基线方法。
原文摘要 · Abstract (English)
Vision-language models (VLMs) have been proven effective for detecting multi-modal misinformation on social platforms, especially in zero-shot settings with unavailable or delayed annotations. However, a single VLM's capacity falls short in the more complex mixed-source multi-modal misinformation detection (M3D) task. Taking captioned images as an example, in M3D, false information can originate from untruthful texts, forged images, or mismatches between the two modalities. Although recent agentic systems can handle zero-shot M3D by connecting modality-specific VLM agents, their effectiveness is still bottlenecked by their architecture. In existing agentic M3D solutions, for any input sample, each agent performs only one forward reasoning pass, making decisions prone to model randomness and reasoning errors in challenging cases. Moreover, the lack of exploration over alternative reasoning paths prevents modern VLMs from fully utilizing their reasoning capacity. In this work, we present AgentM3D, a multi-agent framework for zero-shot M3D. To amplify the reasoning capability of VLMs, we introduce an adaptive test-time scaling paradigm in which each modality-specific VLM agent applies a Best-of-N mechanism, coupled with a critic agent for task-aligned scoring. The agents are organized in a cascading, modality-specific decision chain to reduce unnecessary computation and limit error propagation. To ensure scalability, a planning agent dynamically determines the maximum number of reasoning paths based on sample difficulty, and an adaptive stopping mechanism prevents excessive reasoning within each agent. Extensive experiments on two M3D benchmarks demonstrate that AgentM3D achieves state-of-the-art zero-shot detection performance compared with various VLM-based and agentic baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。