提出三重时域框架,区分模型流式生成的稳定性与长期记忆能力。
Separating Stream Stability from Long-Term Recall in Language Models
- 定义稳定、可访问、有用三重时域,分离流式稳定性与长期记忆
- 注意力缓存可无限稳定生成,但仅保留近期内容,超时即失效
- 适用于评估常量内存下的长序列建模,适合系统设计者参考
流式语言模型的方法常与长上下文和记忆系统混谈,但解决的是不同问题。注意力缓存可在无限长序列上保持自回归生成的稳定性,但模型无法使用已离开最近令牌缓存的内容。我们主张应在系统声明和评估中明确这一区别。引入三个时域:稳定时域(预测行为保持良好)、可访问时域(过去内容仍能因果影响输出)、有用时域(任务性能仍可接受)。我们构造性证明稳定时域可无限,而可访问和有用时域为有限。随后提出ThreeH评估协议,在相同状态与计算预算下测量三者。应用于注意力缓存流式处理,揭示其具备常量内存、稳定生成的优势,但不将锚定标记视为语义记忆。该框架揭示了缓存策略、循环状态、检索与外部记忆的作用。128K token 流、延迟绑定回忆与延迟决策实验表明,注意力缓存保持局部建模,但无法保留活跃缓存之外的内容;循环与检索状态可扩展语义时域。
原文摘要 · Abstract (English)
Methods for streaming language models are often discussed alongside long-context and memory systems, although they solve different problems. An attention sink can stabilize autoregressive generation over an indefinitely long stream while the model remains unable to use content that has left its recent-token cache. We argue that this distinction should be explicit in system claims and evaluation. We introduce three horizons: the stability horizon, over which predictive behavior remains well behaved; the access horizon, over which past content can still causally affect the output; and the utility horizon, over which a task retains acceptable performance. We show constructively that the stability horizon can be infinite while the access and utility horizons are finite. We then propose ThreeH, an evaluation contract that measures all three horizons under a common state and compute budget. Applying the framework to attention-sink streaming clarifies its strength, constant-memory, stable generation, without treating anchor tokens as semantic memory. The framework exposes roles for cache policies, recurrent state, retrieval, and external memory. Experiments on 128K-token streams, delayed binding recall, and delayed decisions show that attention sinks preserve local modeling but not content beyond the active cache; recurrent and retrieval state extend the semantic horizon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。