arXiv:2501.12431cs.LGcs.AI2025-01中稿 · the Proceedings of…被引 36

通过模态交互门控机制,提升图文虚假新闻检测精度。

Modality Interactive Mixture-of-Experts for Fake News Detection

  • 设计分层专家混合框架,显式建模文本与图像的交互关系。
  • 在多语言真实数据集上优于现有方法,显著提升检测准确率。
  • 适合关注虚假信息检测、多模态融合的研究者和应用开发者。

社交媒体上虚假新闻的泛滥对弱势群体造成严重影响,破坏信任、加剧不平等并放大有害叙事。在图文混合情境下检测虚假新闻尤为困难,因文本与图像间存在复杂的相互作用——可能互补、矛盾或独立影响内容真实性。现有方法多强调跨模态一致性,却忽视了模态间的深层互动。为此,本文提出模态交互式专家混合模型(MIMoE-FND),通过交互门控机制显式建模文本与图像的关联性,重点分析单模态预测一致性与语义对齐程度。其分层结构支持针对不同融合场景的差异化学习路径,适配各类模态交互特性。在涵盖两种语言的三个真实世界基准数据集上评估显示,MIMoE-FND性能超越当前最优方法。该模型提升了虚假新闻检测的准确性与可解释性,为遏制虚假信息传播提供了有力工具,有助于保护弱势群体免受其害。

原文摘要 · Abstract (English)

The proliferation of fake news on social media platforms disproportionately impacts vulnerable populations, eroding trust, exacerbating inequality, and amplifying harmful narratives. Detecting fake news in multimodal contexts -- where deceptive content combines text and images -- is particularly challenging due to the nuanced interplay between modalities. Existing multimodal fake news detection methods often emphasize cross-modal consistency but ignore the complex interactions between text and visual elements, which may complement, contradict, or independently influence the predicted veracity of a post. To address these challenges, we present Modality Interactive Mixture-of-Experts for Fake News Detection (MIMoE-FND), a novel hierarchical Mixture-of-Experts framework designed to enhance multimodal fake news detection by explicitly modeling modality interactions through an interaction gating mechanism. Our approach models modality interactions by evaluating two key aspects of modality interactions: unimodal prediction agreement and semantic alignment. The hierarchical structure of MIMoE-FND allows for distinct learning pathways tailored to different fusion scenarios, adapting to the unique characteristics of each modality interaction. By tailoring fusion strategies to diverse modality interaction scenarios, MIMoE-FND provides a more robust and nuanced approach to multimodal fake news detection. We evaluate our approach on three real-world benchmarks spanning two languages, demonstrating its superior performance compared to state-of-the-art methods. By enhancing the accuracy and interpretability of fake news detection, MIMoE-FND offers a promising tool to mitigate the spread of misinformation, with the potential to better safeguard vulnerable communities against its harmful effects.

虚假新闻检测多模态融合专家混合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。