用现代大模型重审联邦党人文集作者归属，发现小模型反而更准。
From Small to Large Language Models: Revisiting the Federalist Papers
- 对比大小模型的嵌入表示，研究文本风格识别方法
- 发现主题词嵌入比词向量扩展更利于作者归属判断
- 适合关注可解释性与经典统计方法的学者
长期以来，联邦党人文集的作者归属问题一直是语言学家、历史学家和统计学家关注的焦点。Mosteller和Wallace（1963）首次使用贝叶斯分析为所有有争议的文章归因于麦迪逊，被视为首个贝叶斯案例研究。本文从现代语言模型的视角重新审视这一经典数据集，涵盖小型和大型语言模型。我们评估了多种主流大语言模型在文本分类中的表现，并探讨其无需微调时的通用嵌入是否适用于风格分析与作者归属。我们比较不同词/短语嵌入方式，讨论如何聚合文档级表示。出乎意料的是，词嵌入维度扩展并不总优于主题嵌入的降维处理。实验表明，即使经过手动微调，默认的大模型嵌入也未能持续提升归属准确率。相反，基于功能词训练的主题嵌入结合贝叶斯分析，在外部样本上表现出更优的分类性能。这说明传统小规模统计语言模型凭借其可解释性和坚实的理论基础，在作者归属任务中仍具显著优势。相关代码已开源：github.com/sowonjeong/slm-to-llm。
原文摘要 · Abstract (English)
For a long time, the authorship of the Federalist Papers had been a subject of inquiry and debate, not only by linguists and historians but also by statisticians. In what was arguably the first Bayesian case study, Mosteller and Wallace (1963) provided the first statistical evidence for attributing all disputed papers to Madison. Our paper revisits this historical dataset but from a lens of modern language models, both small and large. We review some of the more popular Large Language Model (LLM) tools and examine them from a statistical point of view in the context of text classification. We investigate whether, without any attempt to fine-tune, the general embedding constructs can be useful for stylometry and attribution. We explain differences between various word/phrase embeddings and discuss how to aggregate them in a document. Contrary to our expectations, we exemplify that dimension expansion with word embeddings may not always be beneficial for attribution relative to dimension reduction with topic embeddings. Our experiments demonstrate that default LLM embeddings (even after manual fine-tuning) may not consistently improve authorship attribution accuracy. Instead, Bayesian analysis with topic embeddings trained on ``function words" yields superior out-of-sample classification performance. This suggests that traditional (small) statistical language models, with their interpretability and solid theoretical foundation, can offer significant advantages in authorship attribution tasks. The code used in this analysis is available at github.com/sowonjeong/slm-to-llm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。