用靶向帕累托优化提升多模态假新闻检测融合效果
MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection
- 设计分层融合网络，分三阶段进行多模态信息融合
- 在FakeSV和FVC数据集上分别提升2.40%和1.89%准确率
- 适合关注多模态融合与假新闻检测的研究者
多模态假新闻检测对维护互联网多媒体信息真实性至关重要。多模态信息在形式与内容上的显著差异导致优化冲突加剧，阻碍模型有效训练，并降低现有双模态融合方法的效果。为此，我们提出MTPareto框架，采用靶向帕累托（TPareto）优化算法，针对融合层级特定目标进行学习。基于设计的分层融合网络，算法定义三个融合层级及对应损失，并对每个层级实施全模态导向的帕累托梯度融合。该方法通过中间融合获取的信息正向影响整体过程，实现更优的多模态融合。在FakeSV和FVC数据集上的实验结果表明，所提框架优于基线方法，且TPareto优化算法分别实现2.40%和1.89%的准确率提升。
原文摘要 · Abstract (English)
Multimodal fake news detection is essential for maintaining the authenticity of Internet multimedia information. Significant differences in form and content of multimodal information lead to intensified optimization conflicts, hindering effective model training as well as reducing the effectiveness of existing fusion methods for bimodal. To address this problem, we propose the MTPareto framework to optimize multimodal fusion, using a Targeted Pareto(TPareto) optimization algorithm for fusion-level-specific objective learning with a certain focus. Based on the designed hierarchical fusion network, the algorithm defines three fusion levels with corresponding losses and implements all-modal-oriented Pareto gradient integration for each. This approach accomplishes superior multimodal fusion by utilizing the information obtained from intermediate fusion to provide positive effects to the entire process. Experiment results on FakeSV and FVC datasets show that the proposed framework outperforms baselines and the TPareto optimization algorithm achieves 2.40% and 1.89% accuracy improvement respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。