arXiv:2509.19352cs.CLcs.AI2025-09EMNLP被引 4

提出分层软提示模型,解决多模态谣言检测中模态缺失问题。

TriSPrompt: A Hierarchical Soft Prompt Model for Multimodal Rumor Detection with Incomplete Modalities

  • 设计三种提示:模态感知、缺失建模、跨视角关联。
  • 在真实数据集上比顶尖方法准确率提升超13%。
  • 适合处理信息不全的社交媒体谣言检测场景。

多模态数据中普遍存在的模态缺失给谣言检测带来挑战。现有方法主要基于完整多模态训练数据学习联合表示,难以应对现实场景中常见的模态缺失问题。本文提出分层软提示模型 extsf{TriSPrompt},融合三类提示:模态感知(MA)提示捕捉特定模态异构信息与可用数据同质特征,辅助模态恢复;模态缺失(MM)提示建模不完整数据中的缺失状态,提升模型对缺失信息的适应性;互视(MV)提示学习主观(文本与图像)与客观(评论)视角间关系,有效识别谣言。在三个真实世界基准上的大量实验表明, extsf{TriSPrompt} 相比最先进方法准确率提升超过13%。代码与数据集见 https://anonymous.4open.science/r/code-3E88。

原文摘要 · Abstract (English)

The widespread presence of incomplete modalities in multimodal data poses a significant challenge to achieving accurate rumor detection. Existing multimodal rumor detection methods primarily focus on learning joint modality representations from \emph{complete} multimodal training data, rendering them ineffective in addressing the common occurrence of \emph{missing modalities} in real-world scenarios. In this paper, we propose a hierarchical soft prompt model \textsf{TriSPrompt}, which integrates three types of prompts, \textit{i.e.}, \emph{modality-aware} (MA) prompt, \emph{modality-missing} (MM) prompt, and \emph{mutual-views} (MV) prompt, to effectively detect rumors in incomplete multimodal data. The MA prompt captures both heterogeneous information from specific modalities and homogeneous features from available data, aiding in modality recovery. The MM prompt models missing states in incomplete data, enhancing the model's adaptability to missing information. The MV prompt learns relationships between subjective (\textit{i.e.}, text and image) and objective (\textit{i.e.}, comments) perspectives, effectively detecting rumors. Extensive experiments on three real-world benchmarks demonstrate that \textsf{TriSPrompt} achieves an accuracy gain of over 13\% compared to state-of-the-art methods. The codes and datasets are available at https: //anonymous.4open.science/r/code-3E88.

谣言检测多模态软提示缺失模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。