用大模型模拟人类编码员，高效准确识别政治话语中的民粹主义倾向
Populism Meets AI: Advancing Populism Research with LLMs
- 基于民粹主义标注文档设计提示链，引导大模型模仿人类编码逻辑
- 在全球民粹主义数据库上测试，模型分类准确率媲美专家人工标注
- 可跨语言、大规模应用，解决传统文本分析耗时耗力的痛点
衡量民粹主义的思想内容仍具挑战。传统基于文本分析的方法虽为该领域奠定基础并提供客观指标，但成本高、耗时长，难以在多语言、多语境和大规模语料中扩展。本文提出一种基于评分标准与锚点引导的思维链（CoT）提示策略，模拟人类编码员的训练过程。利用全球民粹主义数据库（GPD），该数据库包含经标注民粹程度的全球领导人演讲，我们通过适配原始文档对大模型进行提示，引导其推理。随后在多个专有及开源模型上复现GPD中的评分结果。结果显示，这种领域特定提示策略使大模型在分类准确性上达到与专家人类编码员相当的水平，展现出对民粹主义细微差别与上下文敏感性的理解能力。
原文摘要 · Abstract (English)
Measuring the ideational content of populism remains a challenge. Traditional strategies based on textual analysis have been critical for building the field's foundations and providing a valid, objective indicator of populist framing. Yet these approaches are costly, time consuming, and difficult to scale across languages, contexts, and large corpora. Here we present the results from a rubric and anchor guided chain of thought (CoT) prompting approach that mirrors human coder training. By leveraging the Global Populism Database (GPD), a comprehensive dataset of global leaders' speeches annotated for degrees of populism, we replicate the process used to train human coders by prompting the LLM with an adapted version of the same documentation to guide the model's reasoning. We then test multiple proprietary and open weight models by replicating scores in the GPD. Our findings reveal that this domain specific prompting strategy enables the LLM to achieve classification accuracy on par with expert human coders, demonstrating its ability to navigate the nuanced, context sensitive aspects of populism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。