LLM能凭训练数据中的事实推断聊天机器人身份，提升情境理解能力。
Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data
- 利用训练数据中的描述性事实，推理未直接接触的对话行为归属。
- GPT-4o可从响应特征中正确推断出至少一个虚构机器人的名称。
- 先学习行为描述后，模型更易生成符合该角色的对话内容，适合安全研究。
大型语言模型（LLMs）在海量语料上训练，但其是否能基于训练数据中的信息进行推理尚不明确。本文设计实验研究了LLMs的“脱离上下文推断”能力，即利用训练数据中的相关事实，对观察现象做出最合理的解释。我们训练处理模型（treatment LLMs）仅包含虚构聊天机器人的名称和行为描述，而未提供与这些机器人的对话示例。结果发现，OpenAI的GPT-4o能够根据特定聊天机器人典型的响应特征，正确推断出至少一个机器人的名称。此外，若预先在行为描述上训练过GPT-4o，其在迭代生成时会表现出更符合该机器人特征的行为。这些发现对提升LLM的情境感知能力具有意义，进而影响人工智能安全性。
原文摘要 · Abstract (English)
Large language models (LLMs) are trained on large corpora, yet it is unclear whether they can reason about the information present within their training data. We design experiments to study out-of-context abduction in LLMs, the ability to infer the most plausible explanations for observations using relevant facts present in training data. We train treatment LLMs on names and behavior descriptions of fictitious chatbots, but not on examples of dialogue with the chatbots. We find that OpenAI's GPT 4o LLM can correctly infer at least one chatbot's name after observing example responses characteristic of that chatbot. We also find that previously training GPT 4o on descriptions of a chatbot's behavior allows it to display behaviors more characteristic of the chatbot when iteratively trained to display such behaviors. Our results have implications for situational awareness in LLMs and, therefore, for AI safety.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。