实测发现手机跑大模型慢且效果差,小模型才可行。
Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devices
- 用简化应用测试手机、边缘和云端部署的效率
- 40亿参数以下模型才能在手机上运行,但响应超30秒
- 边缘计算可折中延迟与性能,适合对速度要求不高的场景
近期大语言模型(LLMs)的发展推动了将其部署于移动设备以实现无需云端连接的新应用。然而,在资源受限设备上部署LLMs面临的效率限制带来显著挑战。本文开展全面测量研究,评估基于手机、边缘和云端部署的效率权衡。我们实现了AutoLife-Lite——一个简化版的基于LLM的应用,通过分析智能手机传感器数据推断用户位置与活动上下文。实验结果表明:(1)仅小于40亿参数的模型能在高性能移动设备上成功运行,但相比大模型存在质量局限;(2)模型压缩虽能降低硬件需求,但可能导致显著性能下降;(3)在移动设备上获得有意义输出的延迟显著(>30秒),而云服务表现更优(<10秒);(4)边缘部署在延迟与模型能力间提供中间权衡,且在基于CPU与GPU的设置中表现不同。这些发现为系统设计者提供了当前限制与未来方向的重要洞察。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have prompted interest in deploying these models on mobile devices to enable new applications without relying on cloud connectivity. However, the efficiency constraints of deploying LLMs on resource-limited devices present significant challenges. In this paper, we conduct a comprehensive measurement study to evaluate the efficiency tradeoffs between mobile-based, edge-based, and cloud-based deployments for LLM applications. We implement AutoLife-Lite, a simplified LLM-based application that analyzes smartphone sensor data to infer user location and activity contexts. Our experiments reveal that: (1) Only small-size LLMs (<4B parameters) can run successfully on powerful mobile devices, though they exhibit quality limitations compared to larger models; (2) Model compression is effective in lower the hardware requirement, but may lead to significant performance degradation; (3) The latency to run LLMs on mobile devices with meaningful output is significant (>30 seconds), while cloud services demonstrate better time efficiency (<10 seconds); (4) Edge deployments offer intermediate tradeoffs between latency and model capabilities, with different results on CPU-based and GPU-based settings. These findings provide valuable insights for system designers on the current limitations and future directions for on-device LLM applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。