将多模态假新闻检测转为纯文本任务，大幅降低计算开销。
UNITE-FND: Reframing Multimodal Fake News Detection through Unimodal Scene Translation
- 用提示词将图像转为结构化文本，改用纯文本模型处理。
- 92.52%准确率超越现有模型，参数量减少至1/17。
- 适合资源受限场景，提供图像转文本质量评估新标准。
多模态假新闻检测通常需要复杂架构与大量计算资源，难以在实际中部署。本文提出UNITE-FND框架，将多模态假新闻检测重构为单模态文本分类任务。通过六种专用提示策略结合Gemini 1.5 Pro，将视觉内容转化为结构化文本描述，使纯文本模型能有效保留关键视觉信息。为评估该方法，构建了包含55,000样本的Uni-Fakeddit-55k数据集家族，所有样本均经由多模态到单模态转换流程处理。实验表明，UNITE-FND在二分类任务中达到92.52%准确率，优于现有多模态模型，同时计算成本降低超过10倍（以TinyBERT为例：14.5M参数对比当前最优模型250M+）。此外，提出五项新指标评估图像到文本转换质量，确保信息充分保留。结果表明，结构化文本表示可替代直接多模态处理，在几乎不损失精度的前提下实现更高效率与可扩展性，适用于资源受限环境。
原文摘要 · Abstract (English)
Multimodal fake news detection typically demands complex architectures and substantial computational resources, posing deployment challenges in real-world settings. We introduce UNITE-FND, a novel framework that reframes multimodal fake news detection as a unimodal text classification task. We propose six specialized prompting strategies with Gemini 1.5 Pro, converting visual content into structured textual descriptions, and enabling efficient text-only models to preserve critical visual information. To benchmark our approach, we introduce Uni-Fakeddit-55k, a curated dataset family of 55,000 samples each, each processed through our multimodal-to-unimodal translation framework. Experimental results demonstrate that UNITE-FND achieves 92.52% accuracy in binary classification, surpassing prior multimodal models while reducing computational costs by over 10x (TinyBERT variant: 14.5M parameters vs. 250M+ in SOTA models). Additionally, we propose a comprehensive suite of five novel metrics to evaluate image-to-text conversion quality, ensuring optimal information preservation. Our results demonstrate that structured text-based representations can replace direct multimodal processing with minimal loss of accuracy, making UNITE-FND a practical and scalable alternative for resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。