arXiv:2607.13093cs.CRcs.AI2026-07被引 1

分层协同推理框架,兼顾速度、隐私与设备兼容性。

Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

论文配图:Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models
图 1 · 摘自论文原文
  • 端云协同处理,本地保密计算,云端认证推理
  • 延迟降低46.1%,下行数据量减少67.4%,性能接近全云方案
  • 适合资源受限设备,支持加密传输与轻量化部署

设备端大语言模型推理面临响应延迟、硬件资源有限与用户隐私保护三重挑战。完全依赖云端推理虽算力强大,但会暴露用户提示和对话数据;纯本地推理对多数消费级与嵌入式设备不可行。本文提出一种以隐私为核心的端云协同推理框架,基于端点认证的键值缓存机制。本地端负责输入预处理、嵌入计算、自适应特征优化、KV缓存认证、推测解码及低维模型头计算;云端执行认证解码、KV缓存管理、令牌验证与高维词表投影。本地端融合部分输出,应用语言自适应掩码并采样目标词元。所有传输数据与截断逻辑均经量化并使用AES-GCM加密,核心轻量模块、草稿参数与缓存访问策略均保留在本地以防泄露。该框架通过优化流式处理、批处理与量化ONNX部署,支持包括仅CPU、带GPU及嵌入式设备在内的异构终端。评估表明,相比基线分割推理,该框架将每词元延迟降低最高46.1%,下行负载减少最高67.4%,且性能与全云推理相当。

原文摘要 · Abstract (English)

On-device LLM inference faces a trilemma of response latency, limited hardware resources and user privacy. Full cloud inference delivers strong computing power but exposes user prompts and dialogue data, while standalone on-device inference is unfeasible for most consumer and embedded edge devices. This paper presents a privacy-centric edge-cloud collaborative LLM inference framework built on endpoint-authenticated KV cache. Local endpoints handle input preprocessing, embedding computation, adaptive feature optimization, KV cache authentication, speculative decoding and low-dimensional model head calculation, while the cloud conducts authenticated decoder inference, KV cache management, token verification and high-dimensional vocabulary projection. Endpoints fuse partial outputs, apply language-adaptive masking and sample target tokens. All transmitted data and truncated logits are quantized and AES-GCM encrypted for privacy, with core lightweight modules, draft parameters and cache access policies kept local to avoid leakage. The framework supports heterogeneous devices including CPU-only, GPU-equipped and embedded devices via optimized streaming, batching and quantized ONNX deployment. Evaluations demonstrate that the framework reduces per-token latency by up to 46.1\% and downlink payloads by up to 67.4\% over baseline split inference, retaining comparable performance to full cloud inference.

大模型推理隐私保护边缘计算协同架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。