用压缩检索架构让大模型持久记忆，实现可长期运行的智能操作系统。
The Compressor-Retriever Architecture for Language Model OS
- 设计压缩-检索架构，仅用基础模型前向传播实现上下文压缩与恢复。
- 在上下文学习任务中验证有效，支持跨会话状态保持。
- 适合研究通用智能体、长期记忆系统或构建下一代语言模型操作系统的人。
大型语言模型（LLMs）在多模态信息聚合与处理方面取得显著进展，使其能够执行多模态数据查询、工具调用、网络交互和长文档处理等任务，为将LLM从聊天机器人转变为能与现实世界交互的通用智能体铺平道路。本文探索以语言模型为核心组件构建操作系统（OS）的可行性，将其作为处理存储在上下文窗口中的数据的‘中央处理器’（CPU），而上下文窗口则相当于‘内存’（RAM）。实现这一语言模型操作系统的关键挑战在于管理长期上下文并确保会话间状态连续性，这受限于当前基于会话的交互范式及上下文窗口大小限制。为此，我们提出一种模型无关的压缩-检索架构，用于长期上下文管理。与检索增强生成等方法不同，该方法仅使用基础模型的前向函数完成上下文压缩与检索，保证端到端可微分。初步实验表明该架构在上下文学习任务中有效，标志着向完全状态化的语言模型操作系统迈出重要一步。项目代码仓库：https://github.com/gblackout/LM-OS
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have significantly enhanced their capacity to aggregate and process information across multiple modalities, enabling them to perform a wide range of tasks such as multimodal data querying, tool usage, web interactions, and handling long documents. These capabilities pave the way for transforming LLMs from mere chatbots into general-purpose agents capable of interacting with the real world. This paper explores the concept of using a language model as the core component of an operating system (OS), effectively acting as a CPU that processes data stored in a context window, which functions as RAM. A key challenge in realizing such an LM OS is managing the life-long context and ensuring statefulness across sessions, a feature limited by the current session-based interaction paradigm due to context window size limit. To address this, we introduce compressor-retriever, a model-agnostic architecture designed for life-long context management. Unlike other long-context solutions such as retrieval-augmented generation, our approach exclusively uses the base model's forward function to compress and retrieve context, ensuring end-to-end differentiability. Preliminary experiments demonstrate the effectiveness of this architecture in in-context learning tasks, marking a step towards the development of a fully stateful LLM OS. Project repo available at: https://github.com/gblackout/LM-OS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。