用贝叶斯方法让大模型仅凭一次样例就精准判断作者身份。
A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution
- 基于大模型概率输出构建贝叶斯推理框架，捕捉长程文本关联。
- 在IMDb和博客数据集上实现85%准确率，十作者一shot分类。
- 无需微调，适合数据稀缺的司法语言分析场景。
作者归属旨在识别文档的来源或作者。传统方法依赖人工特征，难以捕捉长距离文本关联，效果受限。近期方法利用预训练语言模型的文本嵌入，但需大量标注数据微调，存在数据依赖和可解释性差的问题。大语言模型（LLMs）具备深度推理能力和长程文本关联保持能力，提供了新思路。本研究探索了预训练LLM在单样本作者归属中的应用，结合贝叶斯方法与模型概率输出，计算文本属于某作者过往写作的概率，实现更精细的作者理解。仅使用Llama-3-70B等预训练模型，在IMDb和博客数据集上对十位作者实现85%的单样本分类准确率。该成果为基于LLM的单样本作者分析设定了新基准，拓展了模型在法语语言学中的应用。研究还包含详尽消融实验验证方法有效性。
原文摘要 · Abstract (English)
Authorship attribution aims to identify the origin or author of a document. Traditional approaches have heavily relied on manual features and fail to capture long-range correlations, limiting their effectiveness. Recent advancements leverage text embeddings from pre-trained language models, which require significant fine-tuning on labeled data, posing challenges in data dependency and limited interpretability. Large Language Models (LLMs), with their deep reasoning capabilities and ability to maintain long-range textual associations, offer a promising alternative. This study explores the potential of pre-trained LLMs in one-shot authorship attribution, specifically utilizing Bayesian approaches and probability outputs of LLMs. Our methodology calculates the probability that a text entails previous writings of an author, reflecting a more nuanced understanding of authorship. By utilizing only pre-trained models such as Llama-3-70B, our results on the IMDb and blog datasets show an impressive 85\% accuracy in one-shot authorship classification across ten authors. Our findings set new baselines for one-shot authorship analysis using LLMs and expand the application scope of these models in forensic linguistics. This work also includes extensive ablation studies to validate our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。