arXiv:2605.07325cs.ROcs.AI2026-05中稿 · IEEE RA-L

让大模型在机器人上实时运行,解决长记忆延迟问题。

CSR: Infinite-Horizon Real-Time Policies with Massive Cached State Representations

论文配图:CSR: Infinite-Horizon Real-Time Policies with Massive Cached State Representations
图 1 · 摘自论文原文
  • 用缓存状态表示法实现高效记忆复用,降低推理延迟。
  • 在120K token上下文下,延迟从14.67秒降至0.56秒,提速26倍。
  • 适合需要高频率、长时序决策的机器人和具身智能系统。

将大规模语言模型(LLMs)作为机器人的持续认知引擎,受限于处理长状态历史所需的时间到首个词元(TTFT)延迟。现有方案如RAG或滑动窗口会牺牲全局上下文或带来高昂重计算成本。本文形式化了最小化延迟的最优任务结构,并理论证明前缀稳定性、增量可扩展性和异步状态同步是实现实时性能的必要条件。基于此,提出缓存状态表示(CSR)框架,实现最优键值缓存复用。为在无限时间尺度上维持该特性,进一步设计异步状态同步(ASR)算法,将状态内存淘汰操作移至并行计算资源,消除延迟尖峰。在连接本地GPU服务器的物理机器人上,针对235B参数模型与120K token上下文,CSR将延迟从14.67秒降至0.56秒,提升26倍;在具身智能基准测试中,召回率达0.836(超越RAG的0.459),同时保持类似延迟。ASR在连续10轮淘汰周期中验证了稳定无尖峰的TTFT表现。总体使大模型可作为高频(>2 Hz)、持续运行的具身策略使用。

原文摘要 · Abstract (English)

Deploying massive large language models (LLMs) as continuous cognitive engines for robotics is bottlenecked by the time-to-first-token (TTFT) latency required to process extensive state histories. Existing solutions like RAG or sliding windows compromise global context or incur prohibitive re-computation costs. We formalize the optimal task structure for minimizing latency and theoretically prove that prefix stability, incremental extensibility, and asynchronous state reconciliation are necessary conditions for real-time performance. Building on these proofs, we introduce the Cached State Representation (CSR) framework as the practical instantiation of these properties, ensuring optimal KV-cache reuse. To sustain these properties over infinite horizons, we further propose an Asynchronous State Reconciliation (ASR) algorithm that offloads state memory eviction to a parallel computational resource to eliminate latency spikes. On a physical robot wirelessly connected to an on-premise GPU server, CSR achieves a 26-fold latency reduction (14.67s to 0.56s) for 120K token contexts with a 235B parameter model compared to a standard baseline. On an embodied AI benchmark, we achieve SOTA recall (0.836 vs. 0.459) while maintaining RAG-level latency. ASR is validated to sustain bounded, spike-free TTFT over 10 eviction cycles in continuous real-world operation. Together, CSR and ASR enable massive LLMs to function as continuously operating, high-frequency (> 2 Hz) embodied policies.

机器人大模型延迟优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。