后训练让大模型小说创作越来越平淡。
Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction
- 通过对比不同阶段模型生成的续写，发现后训练压缩了主题、情感和风格变化。
- 专业文学作品的风格差异被压缩得最明显，公共平台作品次之。
- 模型在对齐后趋于统一输出，不再反映原始故事的叙事特色。
大型语言模型能生成流畅的小说，但其创作常被认为缺乏深度。我们探究这种平庸感源于训练过程中的哪个阶段，以及是否对不同类型小说的影响相同。研究构建了覆盖StoryStar（公开平台）、TMAS（提示引导）和《纽约客》（专业文学）三类文本的匹配续写范式，并对比四个OLMo 32B检查点（Base、SFT、DPO、RLVR）与人类文本的生成结果。由于这些检查点共享架构、规模、分词器和预训练数据，实验可有效分离后训练的影响。我们在句级测量三个维度：主题动态性、情感强度和语言多样性。结果显示，后训练显著压缩了动态变化：主题转换趋于一致，高情绪强度减弱为中性，跨故事的风格差异缩小。这一现象称为“叙事扁平化”。该效应在各类故事中方向一致，但差距大小取决于人类基准——专业文学作品的压缩幅度最大，而公共平台和提示引导故事的差距较小，与其人类基线更接近模型默认节奏一致。后训练终点在不同领域趋同，表明对齐过程使模型生成模式不再敏感于源域的叙事质地。
原文摘要 · Abstract (English)
Large language models produce fluent fiction, yet their creative output is widely seen as flat. We ask where this quality originates in the training and whether it affects different domains of human fiction equally. We construct a matched story-continuation paradigm across StoryStar (public-platform), TMAS (prompt-guided), and The New Yorker (professional literary)-and compare continuations from four OLMo 32B checkpoints (Base, SFT, DPO, RLVR) against matched human text. Because these checkpoints share architecture, scale, tokenizer, and pretraining, the design isolates the post-training effect. We measure each continuation along three sentence-level dimensions: thematic motion, affective prevalence, and linguistic diversity. Across all three, post-training compresses dynamic variation: thematic transitions become more uniform, high-intensity emotions give way to neutrality, and stylistic diversity across stories shrinks. We term this progressive loss narrative flattening. The effect is directionally stable across story domains but gap size depends on the human baseline: professional literary fiction is compressed most, while public-platform and prompt-guided stories show smaller gaps, consistent with their human baselines sitting closer to the model's default rhythm. Post-trained endpoints converge across domains, suggesting alignment produces a continuation regime largely insensitive to the source domain's narrative texture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。