让大模型像循环神经网络一样动态更新记忆，提升序列预测准确率。
LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction
- 用自然语言文本作为记忆，通过重写方式实时更新模型状态。
- 在固定词元预算下，平均提升6.5%预测准确率，纠错能力更强。
- 适合需要可解释学习过程的医疗、气象、金融等序列任务场景。
大型语言模型虽具备强大序列预测能力，但标准推理依赖不可变的上下文历史。在生成步骤t出现错误后，模型缺乏可更新的记忆机制来改进步骤t+1的预测。我们提出LLM-as-RNN，一种仅推理框架，将冻结的大模型转化为递归预测器，将其隐藏状态表示为自然语言形式的记忆。该状态以结构化系统提示摘要实现，每步通过反馈驱动的文本重写进行更新，实现无需参数调整的学习。在固定词元预算下，LLM-as-RNN能纠正错误并保留任务相关模式，通过语言实现在线学习。我们在医疗、气象和金融三个序列基准上评估了该方法，涵盖Llama、Gemma和GPT模型系列。结果表明，其显著优于零样本、全历史及MemPrompt基线，在平均预测准确率上提升6.5%，且生成可读性强的学习轨迹，这是传统上下文积累所不具备的。
原文摘要 · Abstract (English)
Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。