只改3%关键神经元，就能有效纠正大模型的讨好行为。
A Few Bad Neurons: Isolating and Surgically Correcting Sycophancy
- 用稀疏自编码器定位最影响行为的神经元
- 仅更新3%神经元，性能媲美全模型微调
- 适合数据少时精准调整模型行为
大型语言模型的行为对齐通常依赖全局微调，易引发分布偏移和可解释性差等问题。本文提出一种靶向对齐方法：利用稀疏自编码器（SAEs）和线性探测器，识别出对目标行为预测力最强的3%前馈神经网络（MLP）神经元，将其解码至残差空间，并通过梯度掩码仅微调这些神经元。在减少模型讨好行为的任务中，该方法在四个基准测试（Syco-Bench、NLP、POLI、PHIL）上达到或超越当前最优表现，使用Gemma-2-2B和9B模型。结果表明，稀疏的神经元级更新提供了一种可扩展且精准的替代方案，即使在数据稀缺情况下仍具有效性。
原文摘要 · Abstract (English)
Behavioral alignment in large language models (LLMs) is often achieved through broad fine-tuning, which can result in undesired side effects like distributional shift and low interpretability. We propose a method for alignment that identifies and updates only the neurons most responsible for a given behavior, a targeted approach that allows for fine-tuning with significantly less data. Using sparse autoencoders (SAEs) and linear probes, we isolate the 3% of MLP neurons most predictive of a target behavior, decode them into residual space, and fine-tune only those neurons using gradient masking. We demonstrate this approach on the task of reducing sycophantic behavior, where our method matches or exceeds state-of-the-art performance on four benchmarks (Syco-Bench, NLP, POLI, PHIL) using Gemma-2-2B and 9B models. Our results show that sparse, neuron-level updates offer a scalable and precise alternative to full-model fine-tuning, remaining effective even in situations when little data is available
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。