通过分析神经元激活模式,实现无需标注的高效语言模型推理选择。
Do LLMs Signal When They're Right? Evidence from Neuron Agreement
- 利用神经元激活稀疏性和跨样本一致性判断答案正确性。
- 在32个词内预测正确性,减少99%生成令牌消耗。
- 适合需要低延迟、高效率推理的场景,如在线问答系统。
大型语言模型常通过采样-评估-集成解码策略提升推理能力,实现无标签增益而无需真实答案。然而,现有方法仅依赖外部输出信号(如词元概率、熵或自评),这些信号在后训练后校准度差。本文基于神经元激活分析内部行为,发现:(1) 外部信号是更丰富内部动态的低维投影;(2) 正确回答在整个生成过程中激活的唯一神经元显著少于错误回答;(3) 正确回答的激活具有更强的跨样本一致性,错误回答则趋于发散。受此启发,提出无监督的神经元一致解码(NAD),仅使用内部信号进行最佳N选一,不依赖可比文本输出。NAD可在前32个生成词内预测正确性,并支持激进提前停止。在数学与科学基准上,其表现相当于多数投票;在开放编码任务中,持续优于平均@64。通过早期剪枝无效轨迹,NAD将令牌使用量降低99%,同时保持生成质量,证明内部信号可为无标签集成解码提供可靠、可扩展、高效的引导。
原文摘要 · Abstract (English)
Large language models (LLMs) commonly boost reasoning via sample-evaluate-ensemble decoders, achieving label free gains without ground truth. However, prevailing strategies score candidates using only external outputs such as token probabilities, entropies, or self evaluations, and these signals can be poorly calibrated after post training. We instead analyze internal behavior based on neuron activations and uncover three findings: (1) external signals are low dimensional projections of richer internal dynamics; (2) correct responses activate substantially fewer unique neurons than incorrect ones throughout generation; and (3) activations from correct responses exhibit stronger cross sample agreement, whereas incorrect ones diverge. Motivated by these observations, we propose Neuron Agreement Decoding (NAD), an unsupervised best-of-N method that selects candidates using activation sparsity and cross sample neuron agreement, operating solely on internal signals and without requiring comparable textual outputs. NAD enables early correctness prediction within the first 32 generated tokens and supports aggressive early stopping. Across math and science benchmarks with verifiable answers, NAD matches majority voting; on open ended coding benchmarks where majority voting is inapplicable, NAD consistently outperforms Avg@64. By pruning unpromising trajectories early, NAD reduces token usage by 99% with minimal loss in generation quality, showing that internal signals provide reliable, scalable, and efficient guidance for label free ensemble decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。