arXiv:2510.03561cs.CLcs.AI2025-10被引 1

RxT让语言模型实时记忆对话,速度更快成本更低。

Reactive Transformer (RxT) -- Stateful Real-Time Processing for Event-Driven Reactive Language Models

  • 用事件驱动取代逐句重算,实时更新固定大小记忆体
  • 对话复杂度从平方级降为线性,长对话延迟大幅降低
  • 适合需要持续记忆的实时交互场景,如客服、助手

Transformer架构已成为大语言模型的主流,但在对话AI中受限于无状态特性和序列长度的平方级计算复杂度(O(L²))。现有模型通过重复处理不断增长的对话历史来模拟记忆,导致长对话时开销和延迟过高。本文提出Reactive Transformer(RxT),通过从数据驱动转向事件驱动范式,实现真正的实时状态化处理。RxT将每轮对话视为离散事件,采用固定大小的短期记忆(STM)系统保持上下文。其运行周期分为两步:生成器基于当前查询和前一记忆状态生成回复;随后,记忆编码器与专用记忆注意力网络异步更新STM,整合完整交互表示。该设计将对话总用户成本从二次方(O(N²·T))降至线性(O(N·T)),其中N为交互次数。通过解耦响应生成与记忆更新,实现低延迟,支持真正实时、有状态且经济可行的长对话。我们在合成数据上进行了概念验证实验,结果表明,与同等规模的无状态基线模型相比,RxT表现更优且推理延迟恒定。

原文摘要 · Abstract (English)

The Transformer architecture has become the de facto standard for Large Language Models (LLMs), demonstrating remarkable capabilities in language understanding and generation. However, its application in conversational AI is fundamentally constrained by its stateless nature and the quadratic computational complexity ($O(L^2)$) with respect to sequence length $L$. Current models emulate memory by reprocessing an ever-expanding conversation history with each turn, leading to prohibitive costs and latency in long dialogues. This paper introduces the Reactive Transformer (RxT), a novel architecture designed to overcome these limitations by shifting from a data-driven to an event-driven paradigm. RxT processes each conversational turn as a discrete event in real-time, maintaining context in an integrated, fixed-size Short-Term Memory (STM) system. The architecture features a distinct operational cycle where a generator-decoder produces a response based on the current query and the previous memory state, after which a memory-encoder and a dedicated Memory Attention network asynchronously update the STM with a representation of the complete interaction. This design fundamentally alters the scaling dynamics, reducing the total user-facing cost of a conversation from quadratic ($O(N^2 \cdot T)$) to linear ($O(N \cdot T)$) with respect to the number of interactions $N$. By decoupling response generation from memory updates, RxT achieves low latency, enabling truly real-time, stateful, and economically viable long-form conversations. We validated our architecture with a series of proof-of-concept experiments on synthetic data, demonstrating superior performance and constant-time inference latency compared to a baseline stateless model of comparable size.

语言模型实时处理状态记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。