大模型答得对，却不懂如何用知识指导对话行为。
The Knowledge-Behaviour Disconnect in LLM-based Chatbots
- 用大模型训练时的通用预测机制，无法建立知识与行为的连接。
- 无论数据多、训练多，这种知识-行为断层都不会消失。
- 现有伦理对齐技术反而可能加剧断层，导致更严重的幻觉。
基于大语言模型的人工对话代理（如ChatGPT）能回答各种问题，且答案常正确，这让我们倾向于认为它们具备知识。但这些模型是否以知识为基础来指导自身对话行为？我指出，这种能力缺失是根本性的，称为“知识-行为断层”。我进一步论证，这一断层源于大模型训练的核心机制——仅通过上下文预测生成文本，无法建立知识与行为之间的因果联系。即使增加训练数据或扩大模型规模，该断层也不会消失。这揭示了大模型的根本局限，并解释了幻觉的根源。此外，我还讨论了伦理层面的类似断层：尽管研究者已提出多种方法试图引导对话行为符合伦理，但这些技术未能真正解决断层问题，反而可能使其恶化。
原文摘要 · Abstract (English)
Large language model-based artificial conversational agents (like ChatGPT) give answers to all kinds of questions, and often enough these answers are correct. Just on the basis of that capacity alone, we may attribute knowledge to them. But do these models use this knowledge as a basis for their own conversational behaviour? I argue this is not the case, and I will refer to this failure as a `disconnect'. I further argue this disconnect is fundamental in the sense that with more data and more training of the LLM on which a conversational chatbot is based, it will not disappear. The reason is, as I will claim, that the core technique used to train LLMs does not allow for the establishment of the connection we are after. The disconnect reflects a fundamental limitation on the capacities of LLMs, and explains the source of hallucinations. I will furthermore consider the ethical version of the disconnect (ethical conversational knowledge not being aligned with ethical conversational behaviour), since in this domain researchers have come up with several additional techniques to influence a chatbot's behaviour. I will discuss how these techniques do nothing to solve the disconnect and can make it worse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。