构建危机社交网络识别数据集,推出轻量高效模型加速救援信息筛选
"Actionable Help" in Crises: A Novel Dataset and Resource-Efficient Models for Identifying Request and Offer Social Media Posts
- 用生成式大模型与人工校验共建10万条危机微博数据集
- 小型化模型在13类任务中比BERT更快更准,最大提速18.6倍
- 专为资源受限环境设计,适合应急响应系统快速部署
危机期间,社交媒体是关键协调工具,但海量帖子包含大量非行动性内容(如情感支持、行为建议或过时信息),给有效分类带来挑战。尽管生成式大模型可实现少样本分类,但计算开销大,难以实时响应;而微调的编码器模型(如BERT)在资源受限环境下仍存在推理延迟问题。现有蒸馏模型(如DistilBERT)也未针对危机场景优化。为此,我们提出两项贡献:一是构建CrisisHelpOffer数据集,包含10.1万条由生成式大模型协同标注并经人工验证的推文,专门用于区分行动性请求与噪声;二是提出首个面向危机场景的微型模型,可在资源受限环境中高效部署。在13个危机分类任务中,我们的微型模型均优于BERT,性能甚至媲美或超过RoBERTa、MPNet和BERTweet,且体积显著缩小、速度大幅提升:中型模型体积小47%,准确率高3.8%,速度提升3.5倍;小型模型体积小68%,准确率提升1.8%,速度达7.7倍;超小型模型体积减少83%,准确率与BERT相当,速度达18.6倍。所有模型均优于现有蒸馏版本,树立新基准。最后,以全球危机为例,分析了部分发展中国家与发达国家的帮助需求与援助行为差异。
原文摘要 · Abstract (English)
During crises, social media serves as a crucial coordination tool, but the vast influx of posts--from "actionable" requests and offers to generic content like emotional support, behavioural guidance, or outdated information--complicates effective classification. Although generative LLMs (Large Language Models) can address this issue with few-shot classification, their high computational demands limit real-time crisis response. While fine-tuning encoder-only models (e.g., BERT) is a popular choice, these models still exhibit higher inference times in resource-constrained environments. Moreover, although distilled variants (e.g., DistilBERT) exist, they are not tailored for the crisis domain. To address these challenges, we make two key contributions. First, we present CrisisHelpOffer, a novel dataset of 101k tweets collaboratively labelled by generative LLMs and validated by humans, specifically designed to distinguish actionable content from noise. Second, we introduce the first crisis-specific mini models optimized for deployment in resource-constrained settings. Across 13 crisis classification tasks, our mini models surpass BERT (also outperform or match the performance of RoBERTa, MPNet, and BERTweet), offering higher accuracy with significantly smaller sizes and faster speeds. The Medium model is 47% smaller with 3.8% higher accuracy at 3.5x speed, the Small model is 68% smaller with a 1.8% accuracy gain at 7.7x speed, and the Tiny model, 83% smaller, matches BERT's accuracy at 18.6x speed. All models outperform existing distilled variants, setting new benchmarks. Finally, as a case study, we analyze social media posts from a global crisis to explore help-seeking and assistance-offering behaviours in selected developing and developed countries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。