评测主流手机芯片上大模型的性能表现，助力本地化部署优化
Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
- 在商用手机芯片上实测轻量级大模型运行表现
- 覆盖吞吐量、延迟、耗电等多维度指标
- 为开发者优化移动端模型提供硬件选型参考
随着大语言模型（LLMs）日益融入工作与日常生活，用户隐私担忧推动了模型本地部署趋势。目前已有多种轻量级LLM（如Gemini Nano、LLAMA2 7B）可在智能手机上本地运行，提升数据控制力。作为新兴应用，我们关注其在商用手机设备上的实际表现。为此，本文开展全面的移动设备测量研究，评估用户关注的指标（如每秒生成词元数、延迟、响应质量）及开发者关心的因素（资源占用、操作系统策略、电池消耗、启动时间）。同时，对主流厂商的移动系统级芯片（SoCs）进行横向对比，揭示其在处理LLM负载时的性能差异，帮助开发者识别并解决移动端应用瓶颈。本研究可为本地化大模型开发及未来移动系统架构设计提供重要参考。
原文摘要 · Abstract (English)
As large language models (LLMs) increasingly integrate into every aspect of our work and daily lives, there are growing concerns about user privacy, which push the trend toward local deployment of these models. There are a number of lightweight LLMs (e.g., Gemini Nano, LLAMA2 7B) that can run locally on smartphones, providing users with greater control over their personal data. As a rapidly emerging application, we are concerned about their performance on commercial-off-the-shelf mobile devices. To fully understand the current landscape of LLM deployment on mobile platforms, we conduct a comprehensive measurement study on mobile devices. While user experience is the primary concern for end-users, developers focus more on the underlying implementations. Therefore, we evaluate both user-centric metrics-such as token throughput, latency, and response quality-and developer-critical factors, including resource utilization, OS strategies, battery consumption, and launch time. We also provide comprehensive comparisons across the mobile system-on-chips (SoCs) from major vendors, highlighting their performance differences in handling LLM workloads, which may help developers identify and address bottlenecks for mobile LLM applications. We hope that this study can provide insights for both the development of on-device LLMs and the design for future mobile system architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。