用文本类型预测大模型激活值，实现高精度风格识别。
Beyond Tokens in Language Models: Interpreting Activations through Text Genre Chunks
- 通过分析模型激活值，预测输入文本的文体类型。
- 在两个数据集上准确率最高达98%和71%。
- 适合对模型可解释性感兴趣的开发者和研究者。
理解大型语言模型（LLMs）对于确保其安全、有益的部署至关重要。这一任务因模型结构难以解释以及无法对所有输出进行人工评估而变得复杂。本文首次提出一个预测框架：基于大模型的激活值，预测用于提示的文本文体。使用Mistral-7B模型和两个数据集，我们证明了仅用scikit-learn分类器即可实现高达98%和71%的F1分数。在两个数据集中，结果均显著优于对照任务，验证了通过浅层学习模型从大模型中提取文本文体信息的可行性。
原文摘要 · Abstract (English)
Understanding Large Language Models (LLMs) is key to ensure their safe and beneficial deployment. This task is complicated by the difficulty of interpretability of LLM structures, and the inability to have all their outputs human-evaluated. In this paper, we present the first step towards a predictive framework, where the genre of a text used to prompt an LLM, is predicted based on its activations. Using Mistral-7B and two datasets, we show that genre can be extracted with F1-scores of up to 98% and 71% using scikit-learn classifiers. Across both datasets, results consistently outperform the control task, providing a proof of concept that text genres can be inferred from LLMs with shallow learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。