根据词元不确定性动态决定大模型推理是否本地执行或卸载，兼顾精度与延迟。
Accuracy-Delay Trade-Off in LLM Offloading via Token-Level Uncertainty
- 基于词元置信度设计动态卸载策略，优先将不确定的请求发送至边缘服务器。
- 在不同用户密度下均实现更低延迟和更高精度，比基线方法更优。
- 适合资源受限的移动设备在边缘计算中高效运行大语言模型。
大型语言模型（LLMs）虽为智能移动服务带来巨大潜力，但对资源受限设备而言计算开销巨大。移动边缘计算（MEC）允许设备将推理任务卸载至边缘服务器（ES），但通信和服务器队列会引入延迟，尤其在多用户环境下更为显著。本文提出一种基于不确定性的动态卸载框架，根据词元级不确定性与资源约束判断是否本地推理或卸载。定义了一种基于边距的词元级不确定性度量，并验证其与模型精度的相关性。基于该度量，设计贪心卸载算法（GOA），通过优先卸载高不确定性查询，在保证精度的前提下最小化延迟。实验表明，GOA在不同用户密度下均能稳定实现良好精度-延迟权衡，优于基准策略，且计算开销可接受。结果证明，GOA是适用于MEC环境下的可扩展高效解法。
原文摘要 · Abstract (English)
Large language models (LLMs) offer significant potential for intelligent mobile services but are computationally intensive for resource-constrained devices. Mobile edge computing (MEC) allows such devices to offload inference tasks to edge servers (ESs), yet introduces latency due to communication and serverside queuing, especially in multi-user environments. In this work, we propose an uncertainty-aware offloading framework that dynamically decides whether to perform inference locally or offload it to the ES, based on token-level uncertainty and resource constraints. We define a margin-based token-level uncertainty metric and demonstrate its correlation with model accuracy. Leveraging this metric, we design a greedy offloading algorithm (GOA) that minimizes delay while maintaining accuracy by prioritizing offloading for highuncertainty queries. Our experiments show that GOA consistently achieves a favorable trade-off, outperforming baseline strategies in both accuracy and latency across varying user densities, and operates with practical computation time. These results establish GOA as a scalable and effective solution for LLM inference in MEC environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。