arXiv:2608.07208cs.CLcs.AI2026-08

用大模型内部激活值测文本概念含量,效果接近微调模型。

Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes

论文配图:Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes
图 1 · 摘自论文原文
  • 通过线性探测和递归特征机提取大模型激活值中的概念信息
  • 线性探针在ESG文本上准确率达99.4%,接近微调模型表现
  • 激活值能捕捉模型知道但未输出的概念内容,适合内容分析场景

现有衡量文本概念含量的方法依赖表面词汇匹配、主题比例或嵌入相似性,仅反映文本用词,而非读者判断。近期研究表明大型语言模型内部知识与输出之间存在差距。本文探讨是否可通过监测冻结的预训练模型激活值来替代特定任务微调以测量概念内容,并比较不同提取方法的效果。我们采用递归特征机器(RFM)算法和线性探测法提取概念信息,与嵌入基线、表面基线及模型自身回答进行对比。实验基于金融领域文本,使用人工标注的环境、社会与治理(ESG)数据集。最佳线性探测器在未进行任何任务微调的情况下,准确率仅比微调分类器低0.6个百分点,且在十二次比较中有十一次优于模型自身回答,表明激活值蕴含了模型知晓但未表达的概念内容。简单探测法持续优于RFM概念向量,后者虽无法实现分类,但提供连续得分,可反映概念强度,其有效性有待带等级标签的数据验证。

原文摘要 · Abstract (English)

Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities. They score the words a text uses, not the judgment a reader forms about it. Recent work has shown that a gap exists in what Large Language Models (LLMs) know internally versus what they express in their response. This paper asks whether that internal knowledge, read by monitoring the activations of frozen, out-of-the-box LLMs, can stand in for task-specific fine-tuning when measuring concept content, and which extraction method reads it best. We extract such measures via the Recursive Feature Machine (RFM) algorithm and via linear probing, and compare these against an embedding baseline, surface baselines, and the same model's own answer to the question. We demonstrate the approach on financial text, a domain studied extensively and served by established annotated resources, using a human-annotated Environmental, Social and Governance (ESG) dataset. The best linear probe comes within 0.6 percentage points of a fine-tuned domain classifier's accuracy without any task-specific fine-tuning, and outscores the same model's own answer to the question in eleven of twelve comparisons, so the activations carry concept content the response does not report. The simple probe consistently beats the RFM concept vectors, which in turn provide what classification alone does not: a continuous score intended to reflect how strongly a concept is present in a text, whose validation awaits graded labels.

概念检测大模型分析线性探测ESG评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。