用知识图谱增强大模型,更好分析真实罪案播客的复杂叙事。
Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models
- 用知识图谱增强大模型,提升对复杂叙事的理解能力。
- 在主题提取、问答准确率上优于普通大模型,抗干扰更强。
- 适合研究叙事结构、冲突信息处理的学者与内容分析师。
叙事数据贯穿各学科,为读者或观众提供对世界的连贯认知。近年来,机器学习与大语言模型(LLMs)在自然语言分析方面取得显著进展,但对复杂叙事弧线及含矛盾信息的叙述仍存在挑战。已有研究表明,通过外部知识库增强的模型可提升准确性与可解释性。本文评估知识图谱(KG)在真实罪案播客分析中的有效性,对比了基于知识图谱的大模型(KGLLM)与传统NLP方法在知识构建、主题建模和情感分析上的表现。KGLLM支持自然语言查询知识库并回答事实问题,我们测试其在对抗性提示下的鲁棒性,以检验其处理矛盾信息的能力。此外,采用经典方法分析文本中传闻使用与情感建构等细微特征,并提出未来方向。结果表明,KGLLM在多项指标上优于普通大模型,更具鲁棒性,且更擅长将文本归纳为主题。
原文摘要 · Abstract (English)
Narrative data spans all disciplines and provides a coherent model of the world to the reader or viewer. Recent advancement in machine learning and Large Language Models (LLMs) have enable great strides in analyzing natural language. However, Large language models (LLMs) still struggle with complex narrative arcs as well as narratives containing conflicting information. Recent work indicates LLMs augmented with external knowledge bases can improve the accuracy and interpretability of the resulting models. In this work, we analyze the effectiveness of applying knowledge graphs (KGs) in understanding true-crime podcast data from both classical Natural Language Processing (NLP) and LLM approaches. We directly compare KG-augmented LLMs (KGLLMs) with classical methods for KG construction, topic modeling, and sentiment analysis. Additionally, the KGLLM allows us to query the knowledge base in natural language and test its ability to factually answer questions. We examine the robustness of the model to adversarial prompting in order to test the model's ability to deal with conflicting information. Finally, we apply classical methods to understand more subtle aspects of the text such as the use of hearsay and sentiment in narrative construction and propose future directions. Our results indicate that KGLLMs outperform LLMs on a variety of metrics, are more robust to adversarial prompts, and are more capable of summarizing the text into topics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。