大模型能区分作者与文风，但方式不同。
Looking for the Inner Music: Probing LLMs' Understanding of Literary Style
- 用三种方法探查高表现模型的风格特征
- 作者风格比文风更易定义，受语法微调影响大
- 代词使用和词序对两类风格均关键
近期研究证明，语言模型可识别比传统文体学认为更短的文学片段的作者。我们复现了作者归属结果，并扩展至新的小说体裁数据集。发现大模型能区分作者与体裁，但机制不同：部分模型依赖记忆，另一些则通过训练学习特征。我们采用三种方法探查一个高性能模型的风格特征，包括直接修改输入文本的句法结构，以及两种分析模型内部的方法。结果表明，作者风格比体裁风格更易界定，且更受细微句法选择和上下文词汇使用的影响；但代词使用和词序对两类风格均有显著作用。
原文摘要 · Abstract (English)
Recent work has demonstrated that language models can be trained to identify the author of much shorter literary passages than has been thought feasible for traditional stylometry. We replicate these results for authorship and extend them to a new dataset measuring novel genre. We find that LLMs are able to distinguish authorship and genre, but they do so in different ways. Some models seem to rely more on memorization, while others benefit more from training to learn author/genre characteristics. We then use three methods to probe one high-performing LLM for features that define style. These include direct syntactic ablations to input text as well as two methods that look at model internals. We find that authorial style is easier to define than genre-level style and is more impacted by minor syntactic decisions and contextual word usage. However, some traits like pronoun usage and word order prove significant for defining both kinds of literary style.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。