让语言模型在不确定时更专注关键神经元，提升推理表现。
Selective Neuron Amplification in Transformer Language Models
- 通过放大任务相关神经元激活，不改参数实现推理优化。
- 模型不确定时效果显著，自信时影响小，减少误判。
- 适合改进大模型在边缘任务上的表现，无需重训练。
大型语言模型在看似已掌握的任务上仍会失败。实验表明，这并非缺乏知识，而是某些内部回路在推理时未被充分激活。我们提出选择性神经元增强（Selective Neuron Amplification, SNA），在不改变模型参数的前提下，通过增强任务相关神经元的影响力来改善推理表现。该方法仅在推理阶段生效，不永久修改模型。SNA主要在模型不确定时起作用，而在模型已有信心时影响较小，说明部分模型失败源于激活不足而非能力缺失。
原文摘要 · Abstract (English)
Large language models often fail on tasks they seem to already understand. In our experiments, this appears to be less about missing knowledge and more about certain internal circuits not being strongly activated during inference. We explore Selective Neuron Amplification, which increases the influence of task relevant neurons without changing the model's parameters. The method works at inference time and does not permanently alter the model. SNA helps mainly when the model is uncertain, while having low effect when the model is already confident. This suggests that some model failures are due to weak activation rather than lack of capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。