探究大模型是否在内部存储语法错误检测机制
Do Language Models Encode Knowledge of Linguistic Constraint Violations?
- 用稀疏自编码器分解激活模式，寻找语法错误相关特征
- 发现多数语法错误无一致激活特征，支持度有限
- 适合对语言模型内部表征感兴趣的读者
大型语言模型（LLMs）在语言任务中表现优异，但其内部生成机制仍不明确。本文检验假设：LLMs 的参数中编码了语法约束违规的表征，且在处理不合语法句子时会被选择性激活。通过稀疏自编码器将多义激活分解为稀疏的单义特征，并引入敏感度评分识别在违规输入上优先激活的特征，实现无需标注的违规特征探测。进一步提出包含三项标准的联合验证框架。结果显示：（1）各项验证标准未在所有语言现象中同时满足；（2）不同类别间无共有的特征。尽管部分现象显示部分因果结构的证据，整体结果对当前模型存在统一语法违规检测器的支持较弱。
原文摘要 · Abstract (English)
Large Language Models (LLMs) achieve strong linguistic performance, yet their internal mechanisms for producing these predictions remain unclear. We investigate the hypothesis that LLMs encode representations of linguistic constraint violations within their parameters, which are selectively activated when processing ungrammatical sentences. To test this, we use sparse autoencoders to decompose polysemantic activations into sparse, monosemantic features and recover candidates for violation-related features. We introduce a sensitivity score for identifying features that are preferentially activated on constraint-violated versus well-formed inputs, enabling unsupervised detection of potential violation-specific features. We further propose a conjunctive falsification framework with three criteria evaluated jointly. Overall, the results are negative in two respects: (1) the falsification criteria are not jointly satisfied across linguistic phenomena, and (2) no features are consistently shared across all categories. While some phenomena show partial evidence of selective causal structure, the overall pattern provides limited support for a unified set of grammatical violation detectors in current LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。