针对边缘云协同大模型推理,提出基于性能分析的配置选择框架
ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving
- 通过设备与模型对齐的性能分析,建模生成吞吐、接受率和功耗
- 最优吞吐量对应最小最快模型,推测长度为K*=2-10;成本与能耗效率均在K=2时最优
- 揭示多目标冲突,适合部署在异构边缘设备的大模型服务系统
推测解码通过将轻量级分词草稿与重型验证分离,实现云边协同的大语言模型(LLM)推理。尽管先前系统已展示性能与成本优势,但实际部署需应对包括草稿模型变体、量化级别、推测长度及异构边缘设备在内的庞大配置空间。本文提出ConfigSpec,一个面向分布式推测式LLM服务的配置选择框架。该框架对边缘设备和草稿-目标对齐进行性能分析,建模起草稿吞吐、接受率与功耗,以评估不同配置下的有效输出、验证成本效率与能效。在三个边缘平台和两个LLM家族上的分析显示,各优化目标存在结构性冲突:首先,有效输出最大化依赖于最小型、最快草稿模型,且推测长度为设备相关的K*=2-10;其次,成本与能耗效率均在K=2时收敛——成本偏好最大草稿模型以提升接受率,而能耗偏好最小模型以降低功耗。这些冲突表明,单一固定配置无法同时优化所有目标,凸显了在分布式边缘-云LLM推理中采用基于性能分析的配置选择的重要性。
原文摘要 · Abstract (English)
Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from heavyweight verification. While prior systems show performance and cost benefits, practical deployment requires navigating a large configuration space spanning draft model variants, quantisation levels, speculative lengths, and heterogeneous edge devices. This paper presents ConfigSpec, a configurationselection framework for distributed speculative LLM serving. ConfigSpec profiles edge devices and draft-target alignment, and models drafting throughput, acceptance rate, and power to evaluate goodput, verification cost efficiency, and energy efficiency across the joint configuration space. Our analysis across three edge platforms and two LLM families reveals structurally conflicting optima. Firstly, goodput is maximised by the smallest, fastest draft model at device-dependent speculative lengths (K*=2-10). Secondly, both cost and energy efficiency converge to K=2 due to a dominant bonus-token effect-with cost favouring the largest drafter for its high acceptance rate and energy favouring the smallest for its low power draw. These conflicts confirm that no single fixed configuration can simultaneously optimise all objectives, underscoring the need for profiling-based configuration selection in disaggregated edge-cloud LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。