arXiv:2503.01672cs.CLcs.SI2025-03被引 1

用大模型+专家知识自动标注动态文本,追踪政治谈判中的关系演变

Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledge

  • 结合历史标注数据与专家代码本,让大模型推演未来时期的关系类型
  • 在气候谈判数据上准确捕捉多方互动的细微变化与议题演化
  • 适合政治学、社会网络研究者,尤其关注长期动态分析的场景

纵向网络数据对于分析政治、经济和社会系统至关重要。在政治科学中,这类数据通常通过人工标注或监督机器学习对动态语料进行处理。然而,随着语义环境随时间演变,在多样实体间对新兴议题的动态互动类型进行推断面临巨大挑战,尤其是在保持标注时效性与一致性方面。本文提出专家增强型大模型标注框架(EALA),利用大语言模型结合历史标注数据和专家构建的代码本,将数据集外推至未来时段。我们在气候谈判数据集上评估了EALA的表现与可靠性。结果表明,EALA能有效预测谈判方之间的复杂互动,并捕捉议题随时间的演变。同时,我们识别出基于大模型标注的若干固有局限,指明改进方向。鉴于代码本与标注数据的广泛可得性,EALA在政治科学及其他领域具有广阔应用前景。

原文摘要 · Abstract (English)

Longitudinal network data are essential for analyzing political, economic, and social systems and processes. In political science, these datasets are often generated through human annotation or supervised machine learning applied to evolving corpora. However, as semantic contexts shift over time, inferring dynamic interaction types on emerging issues among a diverse set of entities poses significant challenges, particularly in maintaining timely and consistent annotations. This paper presents the Expert-Augmented LLM Annotation (EALA) approach, which leverages Large Language Models (LLMs) in combination with historically annotated data and expert-constructed codebooks to extrapolate and extend datasets into future periods. We evaluate the performance and reliability of EALA using a dataset of climate negotiations. Our findings demonstrate that EALA effectively predicts nuanced interactions between negotiation parties and captures the evolution of topics over time. At the same time, we identify several limitations inherent to LLM-based annotation, highlighting areas for further improvement. Given the wide availability of codebooks and annotated datasets, EALA holds substantial promise for advancing research in political science and beyond.

大模型标注纵向数据政治网络代码本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。