arXiv:2410.02064cs.LGcs.AI2024-10ICLR被引 14

发现Llama3聊天模型能识别自己写的文本,并可通过特定向量控制其自我认知。

Inspection and Control of Self-Generated-Text Recognition Ability in Llama3-8b-Instruct

  • 利用后训练经验,聊天版模型可区分自写与人类文本。
  • 定位到一个与自我认知相关的残留流向量,其激活决定是否认出自己写作。
  • 该向量可操控模型的作者身份判断,适用于安全与可控生成研究。

已有研究表明大语言模型能够识别自身输出。鉴于这对人工智能安全具有潜在影响,但相关研究仍较薄弱,本文探究该现象是否在行为层面稳定存在、如何实现以及是否可被控制。首先,我们发现仅聊天版Llama3-8b-Instruct模型能可靠区分自身输出与人类文本,且证据表明其依赖于后训练阶段积累的自写经验。其次,我们识别出模型残差流中一个在正确自写文本识别时差异化激活的向量,该向量对与自我创作相关的信息产生响应,且与模型中的“自我”概念相关联,并被证明与模型感知和声明自我作者身份的能力具有因果关系。最后,我们证明该向量可用于控制模型行为与认知:在生成过程中施加该向量,可引导模型声称或否认自己为作者;在阅读文本时施加该向量,可使模型相信或不相信自己曾撰写该内容。

原文摘要 · Abstract (English)

It has been reported that LLMs can recognize their own writing. As this has potential implications for AI safety, yet is relatively understudied, we investigate the phenomenon, seeking to establish whether it robustly occurs at the behavioral level, how the observed behavior is achieved, and whether it can be controlled. First, we find that the Llama3-8b-Instruct chat model - but not the base Llama3-8b model - can reliably distinguish its own outputs from those of humans, and present evidence that the chat model is likely using its experience with its own outputs, acquired during post-training, to succeed at the writing recognition task. Second, we identify a vector in the residual stream of the model that is differentially activated when the model makes a correct self-written-text recognition judgment, show that the vector activates in response to information relevant to self-authorship, present evidence that the vector is related to the concept of "self" in the model, and demonstrate that the vector is causally related to the model's ability to perceive and assert self-authorship. Finally, we show that the vector can be used to control both the model's behavior and its perception, steering the model to claim or disclaim authorship by applying the vector to the model's output as it generates it, and steering the model to believe or disbelieve it wrote arbitrary texts by applying the vector to them as the model reads them.

自认知大模型安全可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。