让边缘设备协作生成文本,大幅提升多用户推理效率。
Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge

- 用小模型预生成候选词,大模型在边缘服务器批量验证。
- 通过调节生成长度和分配带宽,使总输出速度提升88%。
- 适合资源有限的终端设备与高并发场景下的智能服务。
推测推理(SPIN)最初被设计为加速大型语言模型(LLMs)的高效架构。本文提出将其分布式部署,以实现多用户边缘系统中的协同令牌生成;其优势在于有效平衡资源受限设备与服务器之间的计算负载。由此提出的架构称为多接入推测推理(Multi-SPIN),利用设备端的小型语言模型生成并上传候选令牌草稿,同时边缘服务器并行批量验证这些草稿。由于用户间计算与通信能力存在严重异构性,草稿长度成为影响节点级计算负载和多接入延迟的关键控制变量,从而决定总令牌产出率(goodput)。因此,在频分多址条件下,我们研究了多接入草稿控制问题,即联合优化草稿长度控制与带宽分配以最大化总令牌产出率。考虑两种情形:(1) 用户间采用同质草稿长度以促进服务器端批量处理;(2) 允许异质草稿长度以引入新的产出率提升维度。通过构建分解方法,将复杂优化问题转化为可解子问题,推导出闭式解的高效草稿控制算法。分析表明,在同质情况下,最优带宽分配会补偿计算与通信能力较弱的用户,因批量同步要求;而在异质情况下,则奖励接受率更高的用户,因放宽同步约束。基于Llama-2与Qwen3.5模型对多样任务的实验表明,Multi-SPIN相较无异构性感知基线,最高提升产出率88%。
原文摘要 · Abstract (English)
Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs). In this work, we propose its distributed deployment to enable cooperative token generation in a multiuser edge system; its advantage is to effectively balance computational loads between resource-constrained devices and servers. The resulting architecture, termed Multi-access SPIN (Multi-SPIN), utilizes on-device small language models to generate and upload candidate token drafts, while an edge server operates the LLM to verify them in parallel batches. Given the severe heterogeneity in users' computation and communication capabilities, the draft length emerges as a critical control variable that influences node-level computation loads and multi-access latency, thereby governing the sum token goodput. Consequently, considering frequency-division multiple access, we investigate the problem of multi-access draft control, a joint optimization of draft-length control and bandwidth allocation to maximize sum token goodput. We examine two cases: (1) homogeneous draft lengths across users to facilitate server-side batching, and (2) heterogeneous draft lengths to introduce a new dimension for goodput enhancement. By developing decomposition methods, we reduce these complex optimizations into tractable sub-problems, which allow efficient draft control algorithms to be derived in closed form. Our analysis shows that the optimal bandwidth allocation compensates users with weaker computation-and-communication capabilities in the homogeneous case due to the batching synchronization requirements, whereas its heterogeneous-case counterpart rewards users with higher acceptance rates by relaxing such requirements. Experiments using Llama-2 and Qwen3.5 model pairs across diverse tasks demonstrate that Multi-SPIN improves goodput by up to 88% over heterogeneity-agnostic baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。