用问题预测大模型答对概率，不需生成答案就能判断可信度。
No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
- 基于问题输入后的激活值，训练线性探针预测答案正确性。
- 在70亿参数模型上，预测准确率超黑盒基线，且跨数据集泛化良好。
- 能识别模型说‘不知道’时的自信程度，适合评估模型可靠性。
大语言模型是否能预判自己能否答对？我们提取模型读完问题但尚未生成回答时的激活值，训练线性探针预测后续答案的正确性。在三个开源模型家族（70亿至70亿参数）中，基于通用趣味问答训练的投影方向，能在分布内及多种分布外知识数据集上有效预测正确性，信号超越数据集特异性伪特征，优于黑盒基线与口头置信度表达。预测能力在中间层达到饱和，但在需要数学推理的问题上泛化能力下降。此外，模型选择“我不知道”时，其探针得分显著升高，表明该方向也捕捉了模型的置信度。结合以往通过探针和稀疏自编码器研究的诚实性等行为，本工作为理解大模型内部机制提供了关键洞见。
原文摘要 · Abstract (English)
Do large language models (LLMs) anticipate when they will answer correctly? To study this, we extract activations after a question is read but before any tokens are generated, and train linear probes to predict whether the model's forthcoming answer will be correct. Across three open-source model families ranging from 7 to 70 billion parameters, projections on this "in-advance correctness direction" trained on generic trivia questions predict success in distribution and on diverse out-of-distribution knowledge datasets, indicating a deeper signal than dataset-specific spurious features, and outperforming black-box baselines and verbalised predicted confidence. Predictive power saturates in intermediate layers and, notably, generalisation falters on questions requiring mathematical reasoning. Moreover, for models responding "I don't know", doing so strongly correlates with the probe score, indicating that the same direction also captures confidence. By complementing previous results on truthfulness and other behaviours obtained with probes and sparse auto-encoders, our work contributes essential findings to elucidate LLM internals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。