让手机上的大模型根据需求自动调节大小，快又省资源。
Elastic On-Device LLM Service
- 通过重排神经元和双头小模型，实现模型与提示的弹性调整。
- 在多种延迟要求下，准确率最高提升14.83%，平均提升10.45%。
- 适合需要低延迟、高隐私的移动端AI应用开发者。
在设备端部署大语言模型（LLM）正推动移动AI发展，支持无需担心隐私的UI自动化等应用。当前普遍做法是用单一强大模型处理多种请求，但存在难以满足多样化推理延迟需求的问题。为此，我们提出\sys，一种可弹性伸缩的设备端LLM服务，能同时调节模型和提示维度。其核心包括：(1) 一次性的神经元重排方法，利用Transformer模型内在的排列一致性，以极低运行时切换开销生成高质量子模型；(2) 双头小型语言模型，高效优化提示并协调模型与提示间的弹性适配。我们在多款市售智能手机上实现该服务，并在独立的NLP/移动代理数据集及端到端合成负载下进行评估。在不同服务等级目标（SLO）下，\sys相比7个强基线，在准确率上绝对提升最高达14.83%，平均提升10.45%，切换开销低于1%，内存消耗相当，离线训练仅需<100 GPU小时。
原文摘要 · Abstract (English)
On-device Large Language Models (LLMs) are transforming mobile AI, catalyzing applications like UI automation without privacy concerns. Nowadays the common practice is to deploy a single yet powerful LLM as a general task solver for multiple requests. We identify a key system challenge in this paradigm: current LLMs lack the elasticity to serve requests that have diversified Service-Level Objectives (SLOs) on inference latency. To tackle this, we present \sys, an on-device LLM service that elasticizes both the model and the prompt dimension of a full LLM. It incorporates (1) a one-shot neuron-reordering method, which leverages the intrinsic permutation consistency in transformer models to generate high-quality elasticized sub-models with minimal runtime switching overhead; (2) a dual-head tiny language model, which efficiently and effectively refines the prompt and orchestrates the elastification between model and prompt. We implement such an elastic on-device LLM service on multiple COTS smartphones, and evaluate \sys on both standalone NLP/mobile-agent datasets and end-to-end synthesized traces. On diverse SLOs, \sys outperforms 7 strong baselines in (absolute) accuracy by up to 14.83\% and 10.45\% on average, with <1\% TTFT switching overhead, on-par memory consumption and <100 offline GPU hours.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。