提出GAMED模型,通过多专家解耦提升多模态假新闻检测效果
GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detection
- 用多个并行专家网络解耦不同模态特征,增强信息选择能力
- 在Fakeddit和Yang数据集上超越现有最优模型,准确率显著提升
- 动态调整模态贡献并提升决策可解释性,适合需要透明性的场景
多模态假新闻检测需建模视觉与语言等异构数据源。现有方法依赖跨模态融合与一致性,难以解析各模态对预测的影响,且多基于静态特征建模,难以适应模态间动态变化。本文提出GAMED框架,通过模态解耦生成区分性特征,强化跨模态协同,优化检测性能。GAMED采用多个并行专家网络提炼特征,并预先嵌入语义知识以提升信息筛选与观点共享能力;随后根据各专家意见自适应调整各模态特征分布。此外,引入新型分类机制,动态管理模态贡献,提高决策可解释性。在Fakeddit与Yang数据集上的实验表明,GAMED优于近期主流先进模型。
原文摘要 · Abstract (English)
Multimodal fake news detection often involves modelling heterogeneous data sources, such as vision and language. Existing detection methods typically rely on fusion effectiveness and cross-modal consistency to model the content, complicating understanding how each modality affects prediction accuracy. Additionally, these methods are primarily based on static feature modelling, making it difficult to adapt to the dynamic changes and relationships between different data modalities. This paper develops a significantly novel approach, GAMED, for multimodal modelling, which focuses on generating distinctive and discriminative features through modal decoupling to enhance cross-modal synergies, thereby optimizing overall performance in the detection process. GAMED leverages multiple parallel expert networks to refine features and pre-embed semantic knowledge to improve the experts' ability in information selection and viewpoint sharing. Subsequently, the feature distribution of each modality is adaptively adjusted based on the respective experts' opinions. GAMED also introduces a novel classification technique to dynamically manage contributions from different modalities, while improving the explainability of decisions. Experimental results on the Fakeddit and Yang datasets demonstrate that GAMED performs better than recently developed state-of-the-art models. The source code can be accessed at https://github.com/slz0925/GAMED.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。