arXiv:2512.23310cs.LGcs.AI2025-12中稿 · ACM for presentati…被引 8

让大模型在边缘与云端智能分工,自动适应网络变化,提速降耗。

Splitwise: Collaborative Edge-Cloud Inference for LLMs via Lyapunov-Assisted DRL

  • 用强化学习动态拆分模型,细粒度控制每层注意力和前馈模块
  • 比现有方法降低1.4到2.8倍延迟,能耗最多减少41%
  • 适合资源受限设备部署,抗网络波动,支持故障快速恢复

将大语言模型(LLMs)部署在边缘设备面临内存和功耗限制。纯云端推理虽减轻设备负担,却带来高延迟和成本。静态划分方式仅优化单一指标,难以应对带宽波动。本文提出Splitwise,一种基于李雅普诺夫优化的深度强化学习框架,实现大模型在边缘与云端的细粒度、自适应划分。该方法将Transformer层分解为注意力头和前馈子模块,提供比传统按层划分更多决策选项。通过分层强化学习策略,在随机工作负载和可变网络条件下,联合最小化延迟、能耗与精度损失,并保证队列稳定。系统还引入带指数退避恢复机制的分区检查点,确保通信失败时的鲁棒性。在Jetson Orin NX、Galaxy S23和Raspberry Pi 5上,使用GPT-2(1.5B)、LLaMA-7B和LLaMA-13B进行实验,结果表明Splitwise相较现有划分器,端到端延迟降低1.4–2.8倍,能耗最高下降41%;95百分位延迟相比纯云端执行降低53%–61%,同时保持精度与较低内存占用。

原文摘要 · Abstract (English)

Deploying large language models (LLMs) on edge devices is challenging due to their limited memory and power resources. Cloud-only inference reduces device burden but introduces high latency and cost. Static edge-cloud partitions optimize a single metric and struggle when bandwidth fluctuates. We propose Splitwise, a novel Lyapunov-assisted deep reinforcement learning (DRL) framework for fine-grained, adaptive partitioning of LLMs across edge and cloud environments. Splitwise decomposes transformer layers into attention heads and feed-forward sub-blocks, exposing more partition choices than layer-wise schemes. A hierarchical DRL policy, guided by Lyapunov optimization, jointly minimizes latency, energy consumption, and accuracy degradation while guaranteeing queue stability under stochastic workloads and variable network bandwidth. Splitwise also guarantees robustness via partition checkpoints with exponential backoff recovery in case of communication failures. Experiments on Jetson Orin NX, Galaxy S23, and Raspberry Pi 5 with GPT-2 (1.5B), LLaMA-7B, and LLaMA-13B show that Splitwise reduces end-to-end latency by 1.4x-2.8x and cuts energy consumption by up to 41% compared with existing partitioners. It lowers the 95th-percentile latency by 53-61% relative to cloud-only execution, while maintaining accuracy and modest memory requirements.

边缘计算大模型部署强化学习智能调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。