用大模型识别叙事中不可靠的讲述者，提升文本可信度判断能力。
Classifying Unreliable Narrators with Large Language Models
- 基于叙事学理论构建三类不可靠叙述分类标准
- 在博客、评论等真实文本上测试大模型效果，挑战显著
- 开源标注数据集与代码，推动可信叙事研究
在第一人称叙述中，我们常需判断讲述者是否可靠。本文提出使用计算方法识别不可靠叙述者，即无意中歪曲信息的讲述者。借鉴叙事学理论，定义了基于文本现象的不可靠叙述类型，构建了TUNa数据集，涵盖博客、Reddit帖子、酒店评论及文学作品等多领域文本。设计了三类分类任务：叙述内、叙述间和跨文本不可靠性。评估了主流开源与专有大模型在此任务上的表现，并探索了少样本、微调和课程学习等多种训练策略。结果表明该任务极具挑战性，但大模型具备识别不可靠叙述的潜力。论文发布专家标注的数据集与代码，欢迎后续研究。
原文摘要 · Abstract (English)
Often when we interact with a first-person account of events, we consider whether or not the narrator, the primary speaker of the text, is reliable. In this paper, we propose using computational methods to identify unreliable narrators, i.e. those who unintentionally misrepresent information. Borrowing literary theory from narratology to define different types of unreliable narrators based on a variety of textual phenomena, we present TUNa, a human-annotated dataset of narratives from multiple domains, including blog posts, subreddit posts, hotel reviews, and works of literature. We define classification tasks for intra-narrational, inter-narrational, and inter-textual unreliabilities and analyze the performance of popular open-weight and proprietary LLMs for each. We propose learning from literature to perform unreliable narrator classification on real-world text data. To this end, we experiment with few-shot, fine-tuning, and curriculum learning settings. Our results show that this task is very challenging, and there is potential for using LLMs to identify unreliable narrators. We release our expert-annotated dataset and code and invite future research in this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。