小模型微调可高效生成领域代码,兼顾速度与精度。
SLM Finetuning for Natural Language to Domain Specific Code Generation in Production
- 用自然语言-代码对微调小模型,嵌入领域知识提升性能。
- 微调后模型在测试集上表现优于大模型,延迟降低40%以上。
- 支持客户定制化微调,适合生产环境部署使用。
当前许多应用使用大语言模型进行代码生成,但生产系统对延迟要求严格,大模型难以满足。参数量几亿的小语言模型资源消耗低,但存在推理能力弱、幻觉多或长上下文记忆差的问题。通过微调将领域知识直接嵌入模型权重,可减少对运行时上下文的依赖。此前我们采用检索增强生成流程,动态选择少样本示例为大模型提供领域上下文。本研究评估了在自然语言-代码对数据集上微调Mistral等小模型生成领域特定代码的效果。结果表明,微调后的模型在测试集上性能和延迟均优于大模型。同时,模型可进一步针对客户场景微调而不影响通用性能,有效解决生产问题。负载测试及生产部署验证了其在延迟与质量上的最优表现。研究证明,小模型通过任务专用微调,是领域代码生成的高效、快速且低成本替代方案。
原文摘要 · Abstract (English)
Many applications today use large language models for code generation; however, production systems have strict latency requirements that can be difficult to meet with large models. Small language models with a few billion parameters are resource efficient but may suffer from limited reasoning, hallucinations, or poor retention of longer context. Fine tuning improves task specific accuracy by embedding domain knowledge directly into model weights, reducing reliance on runtime context. We previously implemented a baseline natural language to code generation approach using a retrieval augmented generation pipeline that dynamically selected few shot examples to embed domain specific language context for a large language model. In this study, we evaluate small language models for generating domain specific language from natural language by fine tuning variants of Mistral and other models on a dataset of natural language code pairs. Our results show that the fine-tuned models achieve improved performance and latency on test datasets compared to larger models. We also demonstrate that the trained model can be further fine-tuned for customer specific scenarios without degrading general performance, helping resolve production issues. Load testing followed by production deployment confirmed optimal performance in terms of latency and quality. These findings demonstrate that task specific fine tuning with small language models provides an efficient, faster, and cost-effective alternative to large language models for domain specific language generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。