通过调控关键神经元提升大模型推理可靠性，无需额外训练。
Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering
- 识别推理正确性相关的少数关键神经元，基于激活差异动态干预。
- 在10个数学与编程基准上实现超13%性能提升，如AIME-24/25。
- 轻量级框架可迁移至不同数据集和更强模型，适合高效推理场景。
尽管近期大语言模型具备强大推理能力，但在复杂任务上实现可靠表现通常需后训练或计算成本高昂的采样策略，限制了实际效率。本文首次发现，大模型中一小部分神经元的激活与推理正确性存在强预测相关性。据此提出轻量级测试时框架AdaRAS（自适应推理激活调控），通过识别推理关键神经元（RCNs）并动态调节其激活值，在不增加训练或采样成本的前提下，提升错误推理路径的正确率，同时避免对已正确案例造成退化。在10个数学与编程基准上的实验表明，该方法实现一致改进，尤其在AIME-24和AIME-25上提升超过13%。此外，AdaRAS展现出良好的跨数据集可迁移性及向更强模型扩展的能力，优于无需额外训练的后训练方法。
原文摘要 · Abstract (English)
Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationally expensive sampling strategies, limiting their practical efficiency. In this work, we first show that a small subset of neurons in LLMs exhibits strong predictive correlations with reasoning correctness. Based on this observation, we propose AdaRAS (Adaptive Reasoning Activation Steering), a lightweight test-time framework that improves reasoning reliability by selectively intervening on neuron activations. AdaRAS identifies Reasoning-Critical Neurons (RCNs) via a polarity-aware mean-difference criterion and adaptively steers their activations during inference, enhancing incorrect reasoning traces while avoiding degradation on already-correct cases. Experiments on 10 mathematics and coding benchmarks demonstrate consistent improvements, including over 13% gains on AIME-24 and AIME-25. Moreover, AdaRAS exhibits strong transferability across datasets and scalability to stronger models, outperforming post-training methods without additional training or sampling cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。