轻量级框架提升灾害信息分类效率，仅用2%参数实现79.6%准确率。
A Lightweight LLM Framework for Disaster Humanitarian Information Classification
- 采用参数高效微调，仅更新2%参数实现高精度分类。
- QLoRA在50%内存下达到99.4%的性能，适合资源受限场景。
- 发现检索增强会因标签噪声降低模型表现，挑战常规认知。
及时分类社交媒体中的灾情信息对有效应急响应至关重要。然而，在资源受限的紧急环境中部署大语言模型面临挑战。本文提出一种轻量、低成本的灾害推文分类框架，采用参数高效微调技术。通过整合并标准化HumAID数据集（覆盖19个灾情事件，共76,484条推文），构建统一双任务基准：人道主义信息分类与事件类型识别。在Llama 3.1 8B模型上系统评估提示策略、LoRA微调及检索增强生成（RAG）效果，结果表明：(1) LoRA实现79.62%的人道主义分类准确率，较零样本提升37.79%，仅训练约2%参数；(2) QLoRA在50%内存开销下达到99.4%的LoRA性能；(3) 与普遍假设相反，RAG因检索样本标签噪声导致微调模型性能下降。这些发现为在有限算力下构建可靠危机情报系统提供了可复现的实用方案。
原文摘要 · Abstract (English)
Timely classification of humanitarian information from social media is critical for effective disaster response. However, deploying large language models (LLMs) for this task faces challenges in resource-constrained emergency settings. This paper develops a lightweight, cost-effective framework for disaster tweet classification using parameter-efficient fine-tuning. We construct a unified experimental corpus by integrating and normalizing the HumAID dataset (76,484 tweets across 19 disaster events) into a dual-task benchmark: humanitarian information categorization and event type identification. Through systematic evaluation of prompting strategies, LoRA fine-tuning, and retrieval-augmented generation (RAG) on Llama 3.1 8B, we demonstrate that: (1) LoRA achieves 79.62% humanitarian classification accuracy (+37.79% over zero-shot) while training only ~2% of parameters; (2) QLoRA enables efficient deployment with 99.4% of LoRA performance at 50% memory cost; (3) contrary to common assumptions, RAG strategies degrade fine-tuned model performance due to label noise from retrieved examples. These findings establish a practical, reproducible pipeline for building reliable crisis intelligence systems with limited computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。