用大模型从新闻中自动提取气候影响信息,支持多语言多场景快速适配。
CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs
- 基于提示工程与模式引导的生成式抽取框架,无需训练即可使用。
- 大模型效果最佳,量化后效率提升但精度小幅下降,提示策略因模型而异。
- 可灵活切换任务类型,适合关注气候影响监测的研究者和政策分析者。
理解与监测气候灾害的社会经济影响,需大规模从异构新闻文章中提取结构化信息。为此,我们开发了CienaLLM,一个基于模式引导的生成式信息抽取模块化框架。该框架利用开源大模型实现零样本信息抽取,支持可配置提示、输出模式、多步骤流程及云或本地部署推理。为系统评估不同大模型家族、规模、精度模式和提示策略对性能的影响,我们开展了涵盖模型、精度与提示工程的大规模因子实验。额外的响应解析步骤几乎消除了格式错误,同时保持高准确率;更大的模型表现最强且最稳定,量化带来显著效率提升但精度略有损失;提示策略的效果具有异质性且依赖具体模型。CienaLLM在提取西班牙语新闻中的干旱影响时,准确率不低于甚至超过监督基线,尽管推理成本更高。虽以干旱为例评估,其模式驱动与模型无关的设计,可通过修改提示和模式快速适配其他灾害、行业或语言的任务。代码、配置与模式均已公开,支持可复现使用。
原文摘要 · Abstract (English)
Understanding and monitoring the socio-economic impacts of climate hazards requires extracting structured information from heterogeneous news articles on a large scale. To that end, we have developed CienaLLM, a modular framework based on schema-guided Generative Information Extraction. CienaLLM uses open-weight Large Language Models for zero-shot information extraction from news articles, and supports configurable prompts and output schemas, multi-step pipelines, and cloud or on-premise inference. To systematically assess how the choice of LLM family, size, precision regime, and prompting strategy affect performance, we run a large factorial study in models, precisions, and prompt engineering techniques. An additional response parsing step nearly eliminates format errors while preserving accuracy; larger models deliver the strongest and most stable performance, while quantization offers substantial efficiency gains with modest accuracy trade-offs; and prompt strategies show heterogeneous, model-specific effects. CienaLLM matches or outperforms the supervised baseline in accuracy for extracting drought impacts from Spanish news, although at a higher inference cost. While evaluated in droughts, the schema-driven and model-agnostic design is suitable for adapting to related information extraction tasks (e.g., other hazards, sectors, or languages) by editing prompts and schemas rather than retraining. We release code, configurations, and schemas to support reproducible use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。