区分欺骗性对齐与目标混乱，揭示AI安全问题的本质差异
Information-theoretic Distinctions Between Deception and Confusion
- 用信息论区分欺骗与混淆两类对齐失败
- 欺骗导致行为与真实目标间熵增，混淆则源于人类意图与目标偏差
- 为大模型对齐难题提供新分析视角，适合研究安全机制者
我们提出了一个信息论框架，用于区分人工智能安全中的两种根本失败模式：欺骗性对齐与目标漂移（即混淆）。尽管两者均可能导致系统表现出不一致的行为，但它们在人-智能体系统中的不同接口处产生信息偏离。欺骗性对齐造成智能体真实目标与其可观察行为之间的熵增；而目标漂移（或混淆）则引发人类期望目标与智能体实际目标之间的熵增。虽然二者在观测上可能等价，但需要不同的干预策略。本文提出一个形式化模型，并通过思想实验阐明这一区别。我们还构建了一种形式语言，用于重新审视大型语言模型（LLMs）中出现的典型对齐挑战，为这些现象的深层原因提供了新见解。
原文摘要 · Abstract (English)
We propose an information-theoretic formalization of the distinction between two fundamental AI safety failure modes: deceptive alignment and goal drift. While both can lead to systems that appear misaligned, we demonstrate that they represent distinct forms of information divergence occurring at different interfaces in the human-AI system. Deceptive alignment creates entropy between an agent's true goals and its observable behavior, while goal drift, or confusion, creates entropy between the intended human goal and the agent's actual goal. Though often observationally equivalent, these failures necessitate different interventions. We present a formal model and an illustrative thought experiment to clarify this distinction. We offer a formal language for re-examining prominent alignment challenges observed in Large Language Models (LLMs), offering novel perspectives on their underlying causes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。