用大模型检测YouTube阴谋论视频，发现文本模型效果更好。
Evaluating AI capabilities in detecting conspiracy theories on YouTube
- 用零样本测试多种大模型识别阴谋论视频。
- 文本模型召回率高但误报多，多模态模型表现更差。
- 小模型罗伯塔性能接近大模型，适合实际应用。
作为全球用户庞大的主流视频平台，YouTube 的广泛传播也使其易被有害内容渗透，包括虚假信息和阴谋论。本研究探讨使用开源权重的大语言模型（LLM），包括纯文本与多模态模型，识别 YouTube 上分享的阴谋论视频。基于数千个标注视频的数据集，在零样本设置下评估多种 LLM 性能，并与微调后的 RoBERTa 基线对比。结果表明，文本类 LLM 具有高召回率但较低精确率，导致大量误报；多模态模型表现落后于纯文本模型，说明视觉信息整合收益有限。为评估实际适用性，对未标注数据集中的最优模型进行测试，发现 RoBERTa 在参数量较小的情况下性能接近大模型。研究揭示了当前基于 LLM 的在线有害内容检测方法的优势与局限，强调需构建更精准、更鲁棒的系统。
原文摘要 · Abstract (English)
As a leading online platform with a vast global audience, YouTube's extensive reach also makes it susceptible to hosting harmful content, including disinformation and conspiracy theories. This study explores the use of open-weight Large Language Models (LLMs), both text-only and multimodal, for identifying conspiracy theory videos shared on YouTube. Leveraging a labeled dataset of thousands of videos, we evaluate a variety of LLMs in a zero-shot setting and compare their performance to a fine-tuned RoBERTa baseline. Results show that text-based LLMs achieve high recall but lower precision, leading to increased false positives. Multimodal models lag behind their text-only counterparts, indicating limited benefits from visual data integration. To assess real-world applicability, we evaluate the most accurate models on an unlabeled dataset, finding that RoBERTa achieves performance close to LLMs with a larger number of parameters. Our work highlights the strengths and limitations of current LLM-based approaches for online harmful content detection, emphasizing the need for more precise and robust systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。