让语言模型像人一样记不住东西，提升用户模拟的可信度
Simulating Human Memory with Language Models
- 用心理学经典实验对比人与语言模型的记忆表现
- 改进提示策略和压缩器，使模型遗忘更像人类
- 在教育任务中验证了类人记忆模型更有效
语言模型被越来越多地用作用户模拟器，但其记忆能力远超真实用户。为衡量这一差距，我们在人类和语言模型上运行了一系列心理学经典记忆实验。结果显示，未经调整的语言模型在各项任务中的记忆表现优于人类，即使被要求模仿人类行为也是如此。随后我们发现，通过优化提示策略并引入压缩器，可使语言模型以更类人的方式遗忘内容。利用这些方法，初步证据表明具有类人记忆限制的语言模型在下游教育任务中能作为更有效的用户模拟器。最后，我们公开了人类参考数据和基准测试集，以支持未来在语言模型中模拟人类记忆的研究。
原文摘要 · Abstract (English)
Language models are increasingly being deployed as user simulators, but their memory is far more reliable than that of real users. To measure this gap, we run a series of classic memory experiments from psychology on both humans and language models. Across tasks, we find that out-of-the-box language models exhibit better memory than humans, even when prompted to imitate human behavior. We then show that better prompting strategies and the use of a compactor can cause language models to forget content in a more human-like way. Using these methods, we show preliminary evidence that language models with human-like memory constraints can function as more effective user simulators in a downstream education task. Finally, we release human reference data and benchmarks to support future work on simulating human memory with language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。