arXiv:2608.10545cs.NIcs.AI2026-08

按重要性传输缓存,500毫秒内实现近全精度的边缘大模型用户切换。

ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover

论文配图:ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover
图 1 · 摘自论文原文
  • 按重要性排序缓存,只传最有价值的部分,降低通信开销。
  • 在500毫秒内达到93.7%平均准确率,接近完整缓存上限。
  • 适用于多用户快速切换场景,尤其适合边缘计算部署。

边缘大模型在用户跨节点切换时需保持推理连续性,依赖键值(KV)缓存迁移至目标节点。然而,同时发生的切换会挤占回传链路带宽,导致无法在移动性限制的传输窗口内完成完整缓存传输。本文不将所有缓存条目视为等值,而是按重要性排序,仅传输最具信息量的部分,将令牌级稀疏性转化为通信节省。将迁移问题建模为多用户回传带宽分配问题,以最大化用户平均准确率为目标。每个用户的部分缓存准确率作为效用函数,其曲线在RULER基准上与测量数据拟合度高达$R^2>0.99$,且准确率曲线凹性覆盖几乎整个缓存范围。所提分配器通过维持服务用户处于该凹区域,使每时隙分配问题变为凸优化,进而导出闭式加权水填解法,支持在线调度。实验表明,在500毫秒传输窗口内,平均准确率达93.7%,距完整缓存上限仅差0.5个百分点,达到类前瞻上限的98.2%-99.5%。

原文摘要 · Abstract (English)

Edge LLMs must preserve inference continuity when a user hands over between edge nodes, requiring key-value (KV) cache transfer to the target node. However, simultaneous handovers saturate the backhaul, preventing full cache delivery within the mobility-imposed transfer window. Rather than allocating bandwidth as if all cache entries were equally valuable, we order each user's KV cache by importance and transmit only its most informative fraction, turning token-level sparsity into communication savings. We cast the transfer as a multi-user backhaul allocation problem that maximizes average accuracy across users. Each user's partial-cache accuracy serves as its utility: a sigmoid that fits measurements on the RULER benchmark with $R^2>0.99$ across models and context lengths. Because importance ordering front-loads the high-value entries, the concave region of the accuracy curve spans nearly the entire cache. Our proposed allocator keeps served users within this region, making each per-slot allocation problem convex. The optimum is derived via a closed-form weighted water-filling solution that generalizes information-theoretic water-filling and enables online scheduling. The proposed allocator attains over 93.7% average accuracy in a 500ms transfer window, within 0.5pp of the full-cache ceiling, and reaches 98.2-99.5% of a clairvoyant upper bound.

边缘计算大模型推理缓存优化多用户调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。