用小模型本地生成物联网代码,更安全、更高效。
GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development
- 微调小模型专注物联网代码生成,支持本地部署。
- 在基准测试中任务准确率提升64.7%,用户满意度显著提高。
- 适合注重隐私与低延迟的嵌入式开发场景。
代码大模型通过自动编写代码和文档提升开发效率,但在需要领域知识的物联网应用中难以生成专用程序。尽管检索增强生成(RAG)可通过获取领域知识改善效果,但依赖强大的云端大模型(如GPT-4)处理需求与检索内容,引发隐私问题,且受网络不稳与高昂查询成本制约,同时难以保证检索内容的正确性与相关性。为此,我们提出GPIoT,一种通过在物联网专用数据集上微调可本地部署的小语言模型(SLMs)来实现物联网程序生成的系统。由于小模型体积小,可在本地高效运行,有效缓解隐私风险与网络不确定性。通过在专有数据集上微调,小模型对物联网程序的生成能力显著增强。为评估其性能,我们构建了基准测试IoTBench。大量实验与用户测试表明,GPIoT在生成物联网代码方面表现优异,相比当前最优代码大模型平均任务准确率提升64.7%,用户满意度明显提高。
原文摘要 · Abstract (English)
Code Large Language Models (LLMs) enhance software development efficiency by automatically generating code and documentation in response to user requirements. However, code LLMs cannot synthesize specialized programs when tasked with IoT applications that require domain knowledge. While Retrieval-Augmented Generation (RAG) offers a promising solution by fetching relevant domain knowledge, it necessitates powerful cloud LLMs (e.g., GPT-4) to process user requirements and retrieved contents, which raises significant privacy concerns. This approach also suffers from unstable networks and prohibitive LLM query costs. Moreover, it is challenging to ensure the correctness and relevance of the fetched contents. To address these issues, we propose GPIoT, a code generation system for IoT applications by fine-tuning locally deployable Small Language Models (SLMs) on IoT-specialized datasets. SLMs have smaller model sizes, allowing efficient local deployment and execution to mitigate privacy concerns and network uncertainty. Furthermore, by fine-tuning the SLMs with our IoT-specialized datasets, the SLMs' ability to synthesize IoT-related programs can be substantially improved. To evaluate GPIoT's capability in synthesizing programs for IoT applications, we develop a benchmark, IoTBench. Extensive experiments and user trials demonstrate the effectiveness of GPIoT in generating IoT-specialized code, outperforming state-of-the-art code LLMs with an average task accuracy increment of 64.7% and significant improvements in user satisfaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。