发现大模型在不确定时会激活相互竞争的特征，揭示了推理过程中的内在机制。
Feature Rivalry in Sparse Autoencoder Representations: A Mechanistic Study of Uncertainty-Driven Feature Competition in LLMs
- 通过负相关特征对研究模型不确定性，定位到特定层的处理阶段。
- 高熵问题下特征竞争显著增强，层0和层12的p值分别低至5.3e-26和5.8e-5。
- 竞争特征方向可控制输出变化，且其得分能预测答案正确性（AUROC=0.689）。
稀疏自编码器（SAEs）将大语言模型的表示分解为可解释特征，但这些特征在不确定性下的交互机制仍不清晰。本文提出‘特征竞争’——即负相关的SAE特征对，并利用Gemma Scope SAEs，在Gemma-2-2B模型上研究其是否作为模型不确定性的机制信号。通过在PopQA数据集按响应熵划分的域内实验，发现高熵问题在层0和层12产生的特征竞争显著强于低熵问题（p值分别为5.3×10⁻²⁶和5.8×10⁻⁵），将不确定性定位到残差流的特定处理阶段。进一步通过沿竞争轴的激活操控实验发现，在低操控强度下，沿竞争方向（vec_A - vec_B）的操控比随机方向引发更大输出变化，覆盖15/20组竞争特征对。最后，基于活跃SAE特征解码向量间余弦相似度计算的每提示竞争得分，可预测答案正确性（AUROC=0.689），接近但未达到Softmax置信度（AUROC=0.808）。
原文摘要 · Abstract (English)
Sparse Autoencoders (SAEs) decompose large language model representations into interpretable features, but how these features interact under uncertainty remains poorly understood. We introduce Feature Rivalry -- negatively correlated SAE feature pairs -- and study whether rivalry serves as a mechanistic signature of model uncertainty in Gemma-2-2B using Gemma Scope SAEs. Through a controlled within-domain experiment on PopQA split by response entropy, we find that high-entropy questions produce significantly stronger feature rivalry at layers 0 and 12 relative to low-entropy questions (p=5.3x10^-26 and p=5.8x10^-5 respectively), localizing uncertainty to specific processing stages in the residual stream. We then test whether rivalry is causally upstream of model outputs via activation steering along rivalry axes -- finding that steering along the rivalry direction (vec_A - vec_B) causes more output changes than random directions at low steering multipliers across 15 of 20 rival feature pairs. Finally, a per-prompt rivalry score derived from pairwise cosine similarities of active SAE feature decoder vectors predicts answer correctness (AUROC=0.689), approaching but not matching softmax confidence (AUROC=0.808).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。