用闭环流程让大模型生成的手机可用神经网络自动优化。
Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search

- 从大模型生成架构到手机实测,全程自动化闭环迭代。
- 在CIFAR-10上移动端得分提升25.6倍,但后续循环无法持续改进。
- 强调真实设备测试必要性,适合移动端模型部署研究者。
将大语言模型(LLM)生成的卷积神经网络部署到真实移动设备,不能仅依赖GPU验证精度:INT8 TensorFlow Lite 导出、硬件加速器选择与设备端延迟共同决定模型是否可用。本文提出一个全自动移动端部署流水线,实现从架构生成模型的QLoRA微调,经GPU评估、INT8导出到真实设备基准测试的闭环,并通过筛选结果动态扩充训练语料。整个流程完全脚本化,支持断点续跑,无需人工干预。在三星SM-P613平板上对CIFAR-10和CIFAR-100两个数据集进行测试(种子42,每轮20个模型,共6轮)。CIFAR-10中第1轮通过门控标准,移动端得分相较基线提升约25.6倍,量化平均准确率达46.9%;后续轮次虽提升GPU精度,但无法维持移动端得分不降。CIFAR-100中,预微调基线保持最佳移动端表现,迭代虽提升GPU精度(最高达26.2%),但未能超越第0轮设备性能,训练池在首次通过后停滞于19个样本。两组实验表明,闭环GPU微调未必带来单调的移动端收益,尤其在更难分类任务上,需结合多数据集与真实设备测量以充分验证部署目标。我们公开每轮的95%置信区间、所有图表及完整复现命令。
原文摘要 · Abstract (English)
Deploying convolutional neural networks generated by large language models (LLMs) on real mobile hardware requires more than GPU validation accuracy: INT8 TensorFlow Lite export, delegate selection, and on-device latency jointly determine whether a model is usable. We present an automated mobile deployment pipeline that closes the loop from QLoRA fine-tuning of an architecture-generating LLM through GPU evaluation, INT8 export, and physical-device benchmarking to gated augmentation of the training corpus. The pipeline is fully scripted and runs cycle-by-cycle without manual intervention, with resume support after interruptions. We evaluate the same frozen protocol on two benchmarks, CIFAR-10 and CIFAR-100, on a Samsung SM-P613 tablet (seed 42, 20 models per cycle, cycles 0-6). On CIFAR-10, cycle 1 is gate-accepted and improves the mobile deployment score approximately 25.6x over the baseline with a mean quantized accuracy of 46.9%; later cycles raise GPU accuracy but fail the non-decreasing mobile gate. On CIFAR-100, the pre-QLoRA baseline retains the best mobile score; iterative rounds improve GPU accuracy (up to 26.2%) yet cannot surpass cycle 0 on-device, and the training pool stalls at 19 examples after the first accepted round. Together, the two studies show that closed-loop GPU fine-tuning does not guarantee monotonic mobile gains, especially on harder classification tasks, and that multi-dataset, on-device measurement is needed to stress-test deployment objectives. We release per-cycle metrics with 95% confidence intervals, all figures, and complete reproduction commands.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。