用微调大模型从微博警情通报中高效提取15项关键信息。
A Domain-Adapted Pipeline for Structured Information Extraction from Police Incident Announcements on Social Media
- 基于LoRA微调Qwen2.5-7B,针对社交媒体文本设计提示工程。
- 在4933条标注数据上实现死亡检测准确率超98.36%,地名提取精确率达95.54%。
- 适合社会科学研究者处理非结构化警情文本,可复用于其他专业领域。
从警方警情通报中提取结构化信息对及时准确的数据处理至关重要,但社交媒体文本的多样性和非正式性带来了显著挑战。为此,我们开发了一种领域适配的抽取流程,采用针对性提示工程与低秩适应(LoRA)微调的Qwen2.5-7B模型,有效应对噪声和异构文本。该方法从27,822条中国微博警情发布帖中构建了高质量、人工标注的4,933个实例数据集,可靠抽取包括地点、事件特征和影响评估在内的15个关键字段。实验表明,基于LoRA的微调显著优于基础模型与指令微调模型,在死亡检测任务上准确率超过98.36%,伤亡人数的精确匹配率达95.31%,省级地点提取精确匹配率为95.54%。该流程为专业领域中的多任务结构化信息抽取提供了经验证的高效解决方案,为社会科学研究中将非结构化文本转化为可靠结构化数据提供了实用框架。
原文摘要 · Abstract (English)
Structured information extraction from police incident announcements is crucial for timely and accurate data processing, yet presents considerable challenges due to the variability and informal nature of textual sources such as social media posts. To address these challenges, we developed a domain-adapted extraction pipeline that leverages targeted prompt engineering with parameter-efficient fine-tuning of the Qwen2.5-7B model using Low-Rank Adaptation (LoRA). This approach enables the model to handle noisy, heterogeneous text while reliably extracting 15 key fields, including location, event characteristics, and impact assessment, from a high-quality, manually annotated dataset of 4,933 instances derived from 27,822 police briefing posts on Chinese Weibo (2019-2020). Experimental results demonstrated that LoRA-based fine-tuning significantly improved performance over both the base and instruction-tuned models, achieving an accuracy exceeding 98.36% for mortality detection and Exact Match Rates of 95.31% for fatality counts and 95.54% for province-level location extraction. The proposed pipeline thus provides a validated and efficient solution for multi-task structured information extraction in specialized domains, offering a practical framework for transforming unstructured text into reliable structured data in social science research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。