用专家标注框架验证大模型从灾情推文提取因果关系的有效性
Large Language Models for Causal Relations Extraction in Social Media: A Validation Framework for Disaster Intelligence
- 构建基于灾情报告的参考图谱，对比大模型生成的因果图
- 发现模型既可识别真实因果，也易受自身先验影响产生虚假关联
- 适合灾情决策支持系统开发者评估大模型可靠性
灾难发生时，从社交媒体中提取因果关系有助于增强态势感知，识别导致伤亡、物理损毁、基础设施中断及连锁影响的因素。然而，灾情相关帖子通常语言非正式、片段化且依赖上下文，可能描述个人经历而非明确因果关系。本文研究大语言模型（LLMs）在灾难社交媒体文本中提取因果关系的有效性。为此，我们提出一个基于专家的评估框架，将模型生成的因果图与基于灾情专项报告生成的参考图进行对比，并评估所提取关系是否得到事件后证据支持，还是仅反映模型先验。研究结果揭示了大模型在灾情决策支持系统中用于因果关系提取的潜力与风险。
原文摘要 · Abstract (English)
During disasters, extracting causal relations from social media can strengthen situational awareness by identifying factors linked to casualties, physical damage, infrastructure disruption, and cascading impacts. However, disaster-related posts are often informal, fragmented, and context-dependent, and they may describe personal experiences rather than explicit causal relations. In this work, we examine whether Large Language Models (LLMs) can effectively extract causal relations from disaster-related social media posts. To this end, we (1) propose an expert-grounded evaluation framework that compares LLM-generated causal graphs with reference graphs derived from disaster-specific reports and (2) assess whether the extracted relations are supported by post-event evidence or instead reflect model priors. Our findings highlight both the potential and risks of using LLMs for causal relation extraction in disaster decision-support systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。