用模型内部状态变化检测答案对错,不依赖训练参数。
CLUE: Non-parametric Verification from Experience via Hidden-State Clustering
- 直接分析模型隐藏状态轨迹,提取正确性几何特征。
- 在AIME 24上将1.5B模型准确率从56.7%提升至70.0%。
- 无需训练参数,适合快速部署到各类大模型验证场景。
评估大语言模型输出质量面临挑战。以往方法依赖文本级信息(如奖励模型、多数投票)或基于词元概率的校准置信度,但前者易受表面线索过拟合,后者在未校准模型上失效。而这些信号其实只是模型内部隐藏状态这一更丰富信息源的部分投影:早期层保留语义与词汇特征,后期层逐渐对齐输出逻辑,蕴含置信度信息。本文探索隐藏状态作为统一验证基础,发现正确性在隐藏激活轨迹中表现为可几何分离的模式。为此提出无参验证器CLUE(Clustering and Experience-based Verification),仅通过隐藏状态差值摘要推理过程,基于最近中心距离分类正确与否,利用历史经验构建成功与失败聚类。该方法简洁却有效,实证表明其持续优于LLM作为裁判基线,并在重排序候选结果上达到或超越现代置信度方法,在AIME 24/25与GPQA上均提升顶1和多数投票准确率。尤其在使用1.5B模型的AIME 24中,准确率从56.7%(多数@64)提升至70.0%(顶多数@16)。
原文摘要 · Abstract (English)
Assessing the quality of Large Language Model (LLM) outputs presents a critical challenge. Previous methods either rely on text-level information (e.g., reward models, majority voting), which can overfit to superficial cues, or on calibrated confidence from token probabilities, which would fail on less-calibrated models. Yet both of these signals are, in fact, partial projections of a richer source of information: the model's internal hidden states. Early layers, closer to token embeddings, preserve semantic and lexical features that underpin text-based judgments, while later layers increasingly align with output logits, embedding confidence-related information. This paper explores hidden states directly as a unified foundation for verification. We show that the correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations. To validate this, we present Clue (Clustering and Experience-based Verification), a deliberately minimalist, non-parametric verifier. With no trainable parameters, CLUE only summarizes each reasoning trace by an hidden state delta and classifies correctness via nearest-centroid distance to ``success'' and ``failure'' clusters formed from past experience. The simplicity of this method highlights the strength of the underlying signal. Empirically, CLUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates, improving both top-1 and majority-vote accuracy across AIME 24/25 and GPQA. As a highlight, on AIME 24 with a 1.5B model, CLUE boosts accuracy from 56.7% (majority@64) to 70.0% (top-maj@16).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。