arXiv:2409.10245cs.CL2024-09NAACL被引 3

用高效微调让大模型自发生成表情符号,展现人格特质。

From Text to Emoji: How PEFT-Driven Personality Manipulation Unleashes the Emoji Potential in LLMs

  • 用量化低秩适配(QLoRA)微调模型,操控五大性格维度。
  • 微调后模型在92.5%至99.5%的测试中自发输出表情符号。
  • 首次通过可解释性方法验证表情符号是模型有意表达人格的表现。

大语言模型(LLMs)的人格操控已成为研究热点。传统方法如基于提示的上下文知识编辑(IKE)和基于梯度的模型编辑网络(MEND)存在不一致和输出混乱的问题:IKE依赖提示导致结果波动,MEND则产生无意义输出。为此,本文采用基于意见问答的参数高效微调(PEFT),具体为量化低秩适配(QLoRA),对开放性、尽责性、外向性、宜人性和神经质这五大人格特质进行操控。微调后,Mistral-7B-Instruct 和 LLaMA-2-7B-chat 等模型展现出在无表情符号训练数据的情况下,仍能生成与特定人格相关的表情符号的潜在行为。例如,LLaMA-2-7B-chat 在 99.5% 的外向性相关测试实例中生成了表情符号,而 Mistral-7B-Instruct 在 92.5% 的开放性相关测试中完成类似表现。上下文学习可解释性分析表明,模型是主动使用表情符号来表达人格;机制可解释性分析显示,该行为可归因于微调后被激活或增强的特定神经元。本文贡献包括:构建首个用于人格操控的 PEFT 意见问答数据集;开发用于评估人格特征的度量模型;证明 PEFT 相较 IKE 在人格操控中的优越性;并通过机制可解释性和上下文学习可解释性方法,分析并验证表情符号的使用动机。

原文摘要 · Abstract (English)

The manipulation of the personality traits of large language models (LLMs) has emerged as a key area of research. Methods like prompt-based In-Context Knowledge Editing (IKE) and gradient-based Model Editor Networks (MEND) have been explored but show irregularity and variability; IKE depends on the prompt, leading to variability and sensitivity, while MEND yields inconsistent and gibberish outputs. To address this, we employed Opinion QA Based Parameter-Efficient Fine-Tuning (PEFT), specifically Quantized Low-Rank Adaptation (QLoRA), to manipulate the Big Five personality traits: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. After PEFT, models such as Mistral-7B-Instruct and LLaMA-2-7B-chat showed a latent behaviour by generating emojis for certain traits, despite no emojis being present in the PEFT data. For instance, LLaMA-2-7B-chat generated emojis in 99.5\% of extraversion-related test instances, while Mistral-7B-Instruct did so in 92.5\% of openness-related test instances. ICL Explainability analysis indicated that the LLMs used emojis intentionally to express these traits. Mechanistic Interpretability analysis showed that this latent behaviour of LLMs could be traced to specific neurons that became activated or amplified after PEFT. This paper provides a number of novel contributions. First, introducing an Opinion QA dataset for PEFT-driven personality manipulation; second, developing metric models to benchmark LLM personality traits; third, demonstrating PEFT's superiority over IKE in personality manipulation; and finally, analysing and validating emoji usage through explainability methods such as Mechanistic Interpretability and In-context learning Explainability methods.

人格操控表情符号高效微调可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。