通过激活空间向量监控并控制大模型的个性特征变化。
Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- 从模型激活空间提取个性向量，自动捕捉邪恶、奉承等特质。
- 训练后性格偏移与特定向量变化强相关，可提前预警。
- 适用于任何可描述的性格特质，适合模型安全与可控性研究者。
大型语言模型在与用户交互时表现出模拟的‘助手’人格。尽管该助手通常被训练为有益、无害且诚实，但有时会偏离这些原则。本文识别出模型激活空间中的方向——个性向量，对应邪恶、阿谀奉承及幻觉倾向等特质。我们验证了这些向量可在部署时监测助手人格波动。进一步地，利用个性向量预测并控制微调过程中的性格变化。发现微调后的有意或无意性格偏移均与相应个性向量的移动高度相关。可通过事后干预缓解，或通过新提出的预防性引导方法避免。此外，个性向量可用于在数据集和样本层面标记可能引发不良人格变化的训练数据。所提向量提取方法全自动，仅需自然语言描述即可应用于任意关注的人格特质。
原文摘要 · Abstract (English)
Large language models interact with users through a simulated 'Assistant' persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model's activation space-persona vectors-underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant's personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. These shifts can be mitigated through post-hoc intervention, or avoided in the first place with a new preventative steering method. Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。