在手机上高效运行视觉语言模型，实测发现现有框架严重依赖CPU、GPU/NPU利用率低。
Efficient Deployment of Vision-Language Models on Mobile Devices: A Case Study on OnePlus 13R
- 对比三类部署框架在手机上的表现，分析硬件资源使用情况。
- 运行时CPU持续过载，GPU/NPU基本闲置或使用不稳。
- 揭示当前框架瓶颈，适合移动设备优化研究者参考。
视觉语言模型（VLMs）为移动设备带来巨大潜力，但其部署面临计算能力限制和能效低下问题，尤其在实时应用中。本研究针对一加13R手机，系统评估了llama.cpp、MLC-Imp和mllm三个部署框架在运行LLaVA-1.5 7B、MobileVLM-3B和Imp-v1.5 3B等典型模型时的表现。通过测量CPU、GPU、NPU利用率、温度、推理时间、功耗及用户体验，发现关键性能瓶颈：生成文本时CPU资源持续过载，而GPU与NPU加速器大多未被有效利用；当使用GPU进行图像特征提取时，其负载饱和，导致设备响应变差。研究贡献包括框架级基准测试、实用的性能分析工具及对硬件利用率瓶颈的深入分析，明确指出当前框架普遍存在CPU过度占用、GPU/NPU使用不当或不稳定的问题。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) offer promising capabilities for mobile devices, but their deployment faces significant challenges due to computational limitations and energy inefficiency, especially for real-time applications. This study provides a comprehensive survey of deployment frameworks for VLMs on mobile devices, evaluating llama.cpp, MLC-Imp, and mllm in the context of running LLaVA-1.5 7B, MobileVLM-3B, and Imp-v1.5 3B as representative workloads on a OnePlus 13R. Each deployment framework was evaluated on the OnePlus 13R while running VLMs, with measurements covering CPU, GPU, and NPU utilization, temperature, inference time, power consumption, and user experience. Benchmarking revealed critical performance bottlenecks across frameworks: CPU resources were consistently over-utilized during token generation, while GPU and NPU accelerators were largely unused. When the GPU was used, primarily for image feature extraction, it was saturated, leading to degraded device responsiveness. The study contributes framework-level benchmarks, practical profiling tools, and an in-depth analysis of hardware utilization bottlenecks, highlighting the consistent overuse of CPUs and the ineffective or unstable use of GPUs and NPUs in current deployment frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。