发现大模型故事生成存在严重多样性缺失，主因是偏好数据影响。
Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories
- 分析4个模型2万条故事，发现11个词出现在88.3%的文本中
- 高频词如Elias、灯塔、钟表匠等在真实文献中罕见，但常见于偏好数据
- 提示小数据集经强对齐算法可能过度影响输出分布，适合关注模型偏见的研究者
大语言模型生成的故事虽流行，但多样性极低。我们从四个主流模型中使用五个提示共采样20,000条故事。结果发现，11个词出现在88.3%的生成故事中，包括名字（Elias、Mara、Elara）、场景（灯塔）和职业（钟表匠、图书管理员）。这些词汇在公开文学作品及预训练数据中并不常见，却频繁出现在偏好数据中，而这类数据很可能被所有当前模型所使用。令人意外的是，包含“灯塔”相关元素的故事在整体生成内容中占比极低，远低于涉及版权角色或成人内容的平均故事。这表明，小规模偏好数据结合强大的对齐算法可能产生不成比例的影响。
原文摘要 · Abstract (English)
LLM-generated stories are a popular use case, but they show very low variability. We sample 20,000 total stories from four current models using five prompts. We find that 11 words occur in 88.3% of generated stories, with little difference between models. These words include names (Elias, Mara, Elara), settings (lighthouses), and professions (clockmaker, librarian). These tokens do not often occur in published literature nor pre-training data, but they are found in preference data that is likely to have been used by all current models. Surprisingly, these "lighthouse" stories are infrequent when compared with the average post-training story, much of which contains references to copyrighted characters or adult content. This result demonstrates the potentially disproportionate impact of small datasets combined with powerful alignment algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。