大模型零样本标注叙事焦点，效果媲美专业人类
Says Who? Effective Zero-Shot Annotation of Focalization
- 用大模型直接分析文学片段的叙事视角限制特征
- GPT-4o平均F1达84.79%，接近人类标注水平
- 可规模化分析小说叙事结构，适合文本研究者
叙事焦点描述叙述者掌握信息的程度如何影响读者获取内容的方式，其编码依赖多种词汇语法特征且需读者解读。即使训练有素的标注员也常对标签存在分歧，表明该任务在定性和计算上均具挑战性。本文测试了五种主流大语言模型家族及两种基线模型在短篇文学片段焦点标注上的表现。尽管任务困难，大模型表现与受训人类标注员相当，其中GPT-4o平均F1达到84.79%。此外，我们发现GPT系列模型输出的对数概率能反映特定片段标注难度。最后，通过对十六部斯蒂芬·金小说的案例研究，展示了该方法在计算文学研究中的实用性，并揭示了大规模分析叙事焦点所能获得的深层洞察。
原文摘要 · Abstract (English)
Focalization describes the way in which access to narrative information is restricted or controlled based on the knowledge available to knowledge of the narrator. It is encoded via a wide range of lexico-grammatical features and is subject to reader interpretation. Even trained annotators frequently disagree on correct labels, suggesting this task is both qualitatively and computationally challenging. In this work, we test how well five contemporary large language model (LLM) families and two baselines perform when annotating short literary excerpts for focalization. Despite the challenging nature of the task, we find that LLMs show comparable performance to trained human annotators, with GPT-4o achieving an average F1 of 84.79%. Further, we demonstrate that the log probabilities output by GPT-family models frequently reflect the difficulty of annotating particular excerpts. Finally, we provide a case study analyzing sixteen Stephen King novels, demonstrating the usefulness of this approach for computational literary studies and the insights gleaned from examining focalization at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。