大模型幻觉信号本质是均值偏移，简单方法就能搞定。
The Hallucination Signal Is a Mean Shift: Why Simple Probes Suffice
- 发现幻觉信号主要由单一均值偏移构成，去除后检测失效。
- 简单线性模型（0.952 AUROC）胜过复杂架构，性能接近最优。
- 信号分布在连续层带，可无需真值层即可聚合到最佳效果。
在三个70亿参数规模的模型和三个数据集的配对样本范式下，我们发现隐藏状态探测器中的幻觉信号主要由单一均值偏移成分主导；移除该方向后检测性能降至随机水平。收缩线性判别分析解决了约73%的一维与全维分类器之间的差距，表明所谓架构复杂性实为高维协方差估计困难所致，而非可利用的非线性。一个简单的L2正则化逻辑回归模型（AUROC 0.952）在性能上可媲美或超越十二种受控架构变体；我们的多层聚合方法在相同范式下优于CLAP跨层注意力探测。由于信号分布于连续层带，LayerMix通过聚合实现无真值层访问下的最优性能。结论限于受控配对样本范式。
原文摘要 · Abstract (English)
Hidden-state probes effectively detect LLM hallucinations, but the geometry of the signal remains poorly characterized, driving increasingly complex probe architectures. Across three 7B-scale models and three datasets in a paired-example paradigm, we find the signal overwhelmingly dominated by a single mean-shift component, and removing this direction collapses detection to chance. Shrinkage linear discriminant analysis closes about 73% of the gap between 1D and full-dimensional classifiers, so apparent architectural complexity largely reflects high-dimensional covariance estimation difficulty rather than exploitable non-linearity. A simple L2-regularized logistic regression (0.952 AUROC) bounds or outperforms twelve controlled architectural alternatives, and our multi-layer aggregation exceeds CLAP cross-layer attention probing under matched paradigm. Because the signal spans a contiguous layer band, LayerMix aggregates it to match oracle-layer performance without oracle access. Our claims characterize the geometry within the controlled paired-example paradigm. Our code is available at https://github.com/js-lee-AI/LayerMix.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。