用AI生成的虚假证据会严重干扰图文错位谣言检测，本文提出两种新方法提升抗干扰能力。
Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection
- 通过跨模态重排序和推理，识别并过滤被AI污染的网络证据。
- 在两个基准数据集上，使现有检测器性能下降不超过9个百分点。
- 适合关注AI造假防御、信息可信度评估的研究者与从业者。
尽管生成式人工智能（GenAI）模型取得显著进展，但其被用于生成误导性内容，正引发对网络信息安全的日益担忧。图文错位（OOC）多模态谣言检测系统通常依赖网页检索的证据来识别被错误引用的图像，但面临越来越严重的GenAI污染证据挑战。现有研究主要针对声明层面的风格重写进行验证，并假设证据库干净。本文打破这一假设，系统研究了GenAI驱动的证据污染对OOC检测的影响。结果表明，被污染的证据可使最先进检测器性能下降超过9个百分点。为此，我们提出两种缓解策略：跨模态证据重排序与跨模态声明-证据推理。在两个基准数据集上的大量实验表明，所提方法能有效增强现有OOC检测器在污染证据下的鲁棒性。源代码与数据已公开于https://github.com/YanZehong/GenAI-Evidence-Pollution。
原文摘要 · Abstract (English)
While generative artificial intelligence (GenAI) models have achieved significant success, their misuse for generating deceptive content raises growing concerns about online information security. Out-of-context (OOC) multimodal misinformation detection systems typically rely on Web-retrieved evidence to identify images repurposed in false contexts, but they are increasingly challenged by the presence of GenAI-polluted evidence. Existing work mainly focuses on verifying claims that have undergone stylistic rewriting at the claim level and assume a clean evidence corpus. In this work, we remove this assumption and systematically study the impact of GenAI-driven evidence pollution threat on OOC detection. We show that polluted evidence can degrade the performance of state-of-the-art detectors by more than 9 percentage points. We propose two mitigating strategies, cross-modal evidence reranking and cross-modal claim-evidence reasoning, to address the challenge posed by polluted evidence. Extensive experiments on two benchmark datasets demonstrate that our approaches effectively enhance the robustness of existing OOC detectors amidst polluted evidence. The source code and data are publicly available at https://github.com/YanZehong/GenAI-Evidence-Pollution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。