提出RACE方法，高效评估大模型神经元在全领域的一致性。
RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons
- 基于前向传播的统计框架，避免梯度计算开销。
- 神经元排序在扰动下展现更强领域特异性效果。
- 计算量仅为梯度法的百分之一，适合大规模分析。
在机制可解释性中，发现跨整个领域的稳定神经元行为仍具挑战。现有方法多依赖实例级点估计或计算昂贵的流程，要么掩盖群体变异性，要么限制全领域可扩展分析。我们提出RACE（Residual Alignment for Consistency Estimation），一种前向传播的统计框架，用于评估Transformer神经元在全领域的功能一致性。相较于基于梯度的点估计，RACE生成的神经元排序在扰动下表现出更显著的领域特异性效应。同时，词元分布偏移支持所选神经元与目标领域的关联性；评分仅需梯度方法约百分之一的计算开销。代码已开源：https://github.com/Nexround/RACE。
原文摘要 · Abstract (English)
Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。