arXiv:2601.06445cs.CLcs.AI2026-01ACL被引 1

构建文学叙事评估基准,揭示大模型在情节结构上的系统性缺陷。

LitVISTA: A Benchmark for Narrative Orchestration in Literary Text

  • 提出VISTA空间统一人类与模型的叙事视角
  • 在金标准事件锚点下,主流模型表现显著不足
  • 适合研究叙事生成、文本理解与模型评估的学者

计算叙事分析旨在捕捉文学文本中的节奏、张力与情感动态。现有大型语言模型虽能生成长篇故事,但过度关注因果连贯性,忽视了人类叙事中复杂的情节弧线与整体编排。这表明模型生成与人类创作在结构上存在偏差。为此,我们提出将叙事分析作为生成能力的诊断指标,构建高维叙事编排框架VISTA Space,融合人类与模型视角,共同表征叙事功能与结构。进一步引入LitVISTA,一个基于文学文本的结构标注基准,实现对模型叙事编排能力的系统评估。在金标准事件锚点的设定下,评测GPT、Claude、Grok和Gemini等前沿模型,结果揭示系统性缺陷:当前模型难以同时把握叙事功能与结构,无法形成整体叙事视图。端到端分析显示,失败主要源于事件锚点识别与定位错误,即便采用先进思维模式,也仅带来有限且不一致的提升。

原文摘要 · Abstract (English)

Computational narrative analysis aims to capture rhythm, tension, and emotional dynamics in literary texts. Existing large language models can generate long stories but overly focus on causal coherence, neglecting the complex story arcs and orchestration inherent in human narratives. This suggests a structural misalignment between model- and human-generated narratives. We therefore position narrative analysis as a diagnostic proxy for generation and propose VISTA Space, a high-dimensional framework for narrative orchestration that unifies human and model perspectives while jointly characterizing narrative function and structure in a common space. We further introduce LitVISTA, a structurally annotated benchmark grounded in literary texts, which operationalizes VISTA Space for systematic evaluation of models' narrative orchestration capabilities. Under an oracle setting with gold event anchors, we evaluate frontier LLMs including GPT, Claude, Grok, and Gemini. Results reveal systematic deficiencies, as current models struggle to jointly capture narrative function and structure and fail to form an integrated global view of literary narrative orchestration. End-to-end analysis further shows that failures are dominated by anchor identification and localization errors. Even advanced thinking modes yield mixed and often limited gains for literary narrative understanding.

叙事生成评估基准大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。