让手机在有限内存下流畅运行大模型，无需修改模型。
Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds
- 通过五项机制拆分模型计算，分散内存压力。
- 单设备峰值内存仅43±2MB，每轮耗电50±3mAh。
- 适合资源受限的移动端部署，尤其安卓设备。
在内存受限的移动设备上部署大型深度神经网络是边缘机器学习的核心挑战。尽管压缩、剪枝和量化能降低参数开销，基于Transformer的模型仍超出主流安卓手机3.3-7.4 GB RAM的内存限制。我们提出CROWDio的DNN流水线调度子系统，无需修改模型即可实现跨资源受限安卓设备的实用ONNX推理，通过五项机制将内存压力分布到多设备：JIT延迟分区加载、单分区驻留约束、四层亲和性调度器、zlib压缩张量传输以及流式1:1依赖模型。在五款安卓手机上对DistilBERT（约6700万参数，SST-2数据集）进行十次运行评估，系统将单设备峰值内存控制在43±2 MB以内，每轮功耗不超过50±3 mAh，同时流式并发使批处理延迟比屏障同步低34%。
原文摘要 · Abstract (English)
Deploying large deep neural networks on memory-constrained mobile devices is a central challenge in edge ML. While compression, pruning, and quantization reduce per-parameter cost, transformer-based models remain too large for the 3.3-7.4 GB RAM envelope of commodity Android handsets. We present the DNN pipeline scheduling subsystem of CROWDio, which achieves practical ONNX inference across resource-constrained Android workers without model modification, by distributing memory pressure across devices via five mechanisms: JIT deferred partition loading, a single-partition-resident constraint, a 4-tier affinity scheduler, a zlib-compressed tensor transport, and a streaming 1:1 dependency model. Evaluated on DistilBERT (Sanh et al., 2019) (approximately 67 M parameters, SST-2) across five Android handsets over ten runs, our system holds peak per-device RSS to 43+-2 MB and limits battery draw to 50+-3 mAh per run, while streaming concurrency cuts batch latency 34% below barrier synchronisation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。