用大模型识别财经新闻中的经济叙事，效果接近专家但仍有差距。
Identifying economic narratives in large text corpora -- An integrated approach using Large Language Models
- 用GPT-4o直接提取新闻中的经济叙事，跳过复杂流水线
- 在通胀相关文章中，模型提取的叙事结构有效但不如专家准确
- 为经济学研究提供大模型应用的新方法和实证参考
近年来，经济叙事研究日益兴起，文本中提取经济叙事的工具也不断增多。现有方法多依赖BERT等先进NLP技术，虽能完成基础语言任务，却缺乏对经济语义深层理解的能力，难以区分经济叙事与常规语义角色标注。本文改用大型语言模型（LLMs），以《华尔街日报》和《纽约时报》关于通货膨胀的新闻文章为语料，采用严格叙事定义，将GPT-4o的输出与专家标注的黄金标准进行对比。结果表明，GPT-4o可在结构化格式下有效提取经济叙事，但在处理复杂文档和深层叙事时仍不及人类专家。鉴于大模型在经济学研究中的新颖性，本文还为未来社会科学领域使用大模型开展类似研究提供了实践指导。
原文摘要 · Abstract (English)
As interest in economic narratives has grown in recent years, so has the number of pipelines dedicated to extracting such narratives from texts. Pipelines often employ a mix of state-of-the-art natural language processing techniques, such as BERT, to tackle this task. While effective on foundational linguistic operations essential for narrative extraction, such models lack the deeper semantic understanding required to distinguish extracting economic narratives from merely conducting classic tasks like Semantic Role Labeling. Instead of relying on complex model pipelines, we evaluate the benefits of Large Language Models (LLMs) by analyzing a corpus of Wall Street Journal and New York Times newspaper articles about inflation. We apply a rigorous narrative definition and compare GPT-4o outputs to gold-standard narratives produced by expert annotators. Our results suggests that GPT-4o is capable of extracting valid economic narratives in a structured format, but still falls short of expert-level performance when handling complex documents and narratives. Given the novelty of LLMs in economic research, we also provide guidance for future work in economics and the social sciences that employs LLMs to pursue similar objectives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。