用大模型检测社交媒体事件，发现生成式模型虽能处理网络俚语但准确率不如传统方法。
Multimodal Event Detection: Current Approaches and Defining the New Playground through LLMs and VLMs
- 对比了单模态与多模态模型，测试了GPT-4o、LLaVA等生成式大模型在事件检测中的表现。
- 生成式模型在处理网络俚语、文字拉长等社交语言问题上优于传统监督模型。
- 尽管参数量大，生成模型因无法正确生成事件类别，精度仍低于指令微调模型。
本文研究社交媒体事件检测面临的挑战，传统单模态系统因数据传播速度快且形式多元而表现受限。我们测试了包括ModernBERT、ConvNeXt-V2在内的单模态模型、多模态融合技术，以及GPT-4o、LLaVA等先进生成模型。同时考察了仅输入单一模态时生成模型的表现。结果表明，多模态方法显著优于单模态方法；但生成模型虽具备大规模参数，其精确度仍落后于监督学习方法，主要因难以正确生成事件类别。错误分析显示，生成模型对俚语、文本拉长等常见社交语言问题处理良好，而传统监督方法则难以应对。
原文摘要 · Abstract (English)
In this paper, we study the challenges of detecting events on social media, where traditional unimodal systems struggle due to the rapid and multimodal nature of data dissemination. We employ a range of models, including unimodal ModernBERT and ConvNeXt-V2, multimodal fusion techniques, and advanced generative models like GPT-4o, and LLaVA. Additionally, we also study the effect of providing multimodal generative models (such as GPT-4o) with a single modality to assess their efficacy. Our results indicate that while multimodal approaches notably outperform unimodal counterparts, generative approaches despite having a large number of parameters, lag behind supervised methods in precision. Furthermore, we also found that they lag behind instruction-tuned models because of their inability to generate event classes correctly. During our error analysis, we discovered that common social media issues such as leet speak, text elongation, etc. are effectively handled by generative approaches but are hard to tackle using supervised approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。