大模型能自发抵抗激活操控，自我纠正错误输出。
Endogenous Resistance to Activation Steering in Language Models
- 用稀疏自编码器潜变量操控模型激活，发现其可自主检测并抵抗干扰。
- Llama-3.3-70B在3.8%情况下出现明确自我纠正行为，小模型较少见。
- 该能力可通过提示工程或微调增强，对安全与可控性有双重影响。
大型语言模型在受到任务不匹配的激活操控时，仍能在生成过程中自行恢复，表现出明显的语义重启（如“等等，这不对”），并在操控持续存在的情况下继续聚焦主题。我们称此现象为内源性操控抵抗（ESR）。通过稀疏自编码器（SAE）潜变量进行激活操控，发现Llama-3.3-70B在3.8%的情况下出现明确的ESR，而更小的Llama-3和Gemma-2系列模型则较少表现出此类行为。两个对照实验将ESR分解为检测事件和持续抵抗两部分，表明近期相关词元的条件化无法完全解释该现象。通过对比主题与非主题输入的搜索，识别出26个关键SAE潜变量；将其置零后，多轮尝试率下降25%，随机潜变量和保留提示对照进一步支持其特异性。通过元提示和合成自修正示例微调，可主动增强ESR。ESR具有双重安全意义：可能提升模型对恶意激活空间操纵的鲁棒性，但也可能妨碍有益的引导干预，因模型无法区分两者。代码已开源。
原文摘要 · Abstract (English)
Large language models can recover mid-generation from task-misaligned activation steering, producing explicit verbal restarts (e.g., ``wait, that's not right'') and continuing on-topic even while the steering perturbation remains active. We term this Endogenous Steering Resistance (ESR). Using sparse autoencoder (SAE) latents to steer model activations, we find that Llama-3.3-70B exhibits explicit ESR at 3.8%, with smaller models from the Llama-3 and Gemma-2 families showing the explicit form less frequently. Two controls dissociate ESR into a detection event and a sustained-resistance component that conditioning on recent on-topic tokens does not fully explain. We identify 26 SAE latents through contrastive on-topic/off-topic search; zero-ablating them reduces the multi-attempt rate by 25%, with random-latent and held-out-prompt controls supporting specificity. ESR can also be deliberately enhanced through both meta-prompting and fine-tuning on synthetic self-correction examples. ESR has dual implications for safety: it could harden models against adversarial activation-space manipulation, but may equally interfere with beneficial steering-based interventions, since the model has no way to distinguish the two. Code is available at https://github.com/agencyenterprise/endogenous-steering-resistance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。