将用户上下文以分词形式分布存储，提升边缘部署大模型的响应速度与一致性。
DisCEdge: Distributed Context Management for Large Language Models at the Edge
- 用分词序列在边缘节点间分布式存储用户上下文
- 相比原始文本系统，响应时间提升14.46%，同步开销降低15%
- 适合对延迟敏感、隐私要求高的边缘AI应用
将大语言模型（LLM）服务部署在边缘可显著提升低延迟和高隐私场景的体验。然而，由于LLM的无状态特性，跨地理分布的边缘节点管理用户上下文（如会话、偏好）存在挑战。现有方案如客户端本地存储会引入网络延迟和带宽开销，削弱边缘部署优势。本文提出DisCEdge，一种分布式上下文管理系统，将用户上下文以分词形式在边缘节点间存储与复制。通过保持上下文为分词序列，系统避免重复计算并实现高效数据复制。我们在真实边缘环境中评估了开源原型，结果表明，相比基于原始文本的系统，DisCEdge将中位响应时间提升最高达14.46%，节点间同步开销降低最多15%；同时相比客户端存储，中位请求尺寸减少90%，且保证数据一致性。
原文摘要 · Abstract (English)
Deploying Large Language Model (LLM) services at the edge benefits latency-sensitive and privacy-aware applications. However, the stateless nature of LLMs makes managing user context (e.g., sessions, preferences) across geo-distributed edge nodes challenging. Existing solutions, such as client-side context storage, introduce network latency and bandwidth overhead, undermining edge deployment advantages. We propose DisCEdge, a distributed context management system that stores and replicates user context in tokenized form across edge nodes. By maintaining context as token sequences, our system avoids redundant computation and enables efficient data replication. We evaluate an open-source prototype in a realistic edge environment. DisCEdge improves median response times by up to 14.46% and lowers median inter-node synchronization overhead by up to 15% compared to a raw-text-based system. It also reduces client request sizes by a median of 90% compared to client-side context management, while guaranteeing data consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。