大模型无法玩猜字游戏,因缺乏私有记忆机制。
LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents
- 定义隐藏状态交互任务,揭示公开对话无法保持秘密的理论瓶颈。
- 实验验证大模型在分叉对话中无法维持秘密,即使扩大规模也失败。
- 提出显式私有工作内存架构,使模型能稳定维护隐藏状态。
随着大语言模型从文本补全转向自主代理,其仍受限于标准聊天界面,缺乏私有工作记忆。这引出一个根本问题:代理能否可靠执行依赖隐藏状态的交互任务?我们定义了私有状态交互任务(PSITs),要求代理在生成公开回应时持续维护隐藏信息。理论上证明,仅依赖公开对话历史的代理无法同时保持秘密未暴露且回应与固定隐藏状态一致,形成架构上的不可能定理。为实证验证此局限性,我们设计自一致性测试协议,评估代理在分叉对话分支中维持隐藏秘密的能力。标准聊天型大模型与基于检索的记忆基线无论规模如何均未能通过测试,表明语义检索无法实现真正状态维护。为此,我们提出一种包含显式私有工作记忆的新架构,实证显示该机制可恢复与固定隐藏状态的一致性,确立私有状态是具备PSIT能力的语言代理的必要组件。代码已开源。
原文摘要 · Abstract (English)
As LLMs move from text completion toward autonomous agents, they remain constrained by the standard chat interface, which lacks private working memory. This raises a fundamental question: can agents reliably perform interactive tasks that depend on hidden state? We define Private State Interactive Tasks (PSITs), which require agents to generate and maintain hidden information while producing public responses consistent with a fixed hidden state. We show theoretically that any agent restricted to the public conversation history cannot both keep the secret unresolved in the transcript and respond consistently with a fixed hidden state in PSITs, yielding an architectural impossibility theorem. To empirically validate this limitation, we introduce a self-consistency testing protocol that evaluates whether agents can maintain a hidden secret across forked dialogue branches. Standard chat-based LLMs and retrieval-based memory baselines fail this test regardless of scale, demonstrating that semantic retrieval does not enable true state maintenance. To address this, we propose a novel architecture incorporating an explicit private working memory; we demonstrate that this mechanism restores consistency with a fixed hidden state, establishing private state as a necessary component for PSIT-capable language agents. Our code is available at https://github.com/chandar-lab/Hangman
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。