arXiv:2412.15441cs.SEcs.AI2024-12中稿 · Journal of Systems…被引 6

选对推理引擎和执行器,能让代码小模型省电超89%。

Insights into resource utilization of code small language models serving with runtime engines and execution providers

  • 对比12个代码生成小模型,测试不同运行时与执行器组合的资源表现。
  • 用CUDA+Torch配置,比其他方案最多省电89.16%,提速且更高效。
  • 适合关注能效、部署性能的开发者,尤其在资源受限场景下。

语言模型尤其是代码生成模型的快速发展带来了巨大的计算资源需求,引发对能耗和环境影响的担忧。优化推理阶段的资源利用至关重要,而小语言模型(SLMs)为此提供了可行路径。本文从软件工程师角度出发,分析深度学习服务配置(即运行时引擎与执行器的组合)对代码生成类小模型在能源消耗、执行时间及计算资源利用率方面的影响。通过多阶段实验,使用12个代码生成小模型系统评估不同配置。结果显示:采用CUDA执行器的配置在能耗和执行时间上均优于CPU配置;其中,Torch搭配CUDA的组合实现最高能效,相比其他配置节能37.99%至89.16%;在仅使用CPU的配置中,优化后的ONNX搭配CPU也实现了8.98%至72.04%的节能效果。此外,Torch+CUDA配置展现出优异的计算资源利用率。服务配置选择显著影响资源效率,建议根据实际需求优先选用上述组合。

原文摘要 · Abstract (English)

The rapid growth of language models, particularly in code generation, requires substantial computational resources, raising concerns about energy consumption and environmental impact. Optimizing language models inference resource utilization is crucial, and Small Language Models (SLMs) offer a promising solution to reduce resource demands. Our goal is to analyze the impact of deep learning serving configurations, defined as combinations of runtime engines and execution providers, on resource utilization, in terms of energy consumption, execution time, and computing-resource utilization from the point of view of software engineers conducting inference in the context of code generation SLMs. We conducted a technology-oriented, multi-stage experimental pipeline using twelve code generation SLMs to investigate energy consumption, execution time, and computing-resource utilization across the configurations. Significant differences emerged across configurations. CUDA execution provider configurations outperformed CPU execution provider configurations in both energy consumption and execution time. Among the configurations, TORCH paired with CUDA demonstrated the greatest energy efficiency, achieving energy savings from 37.99% up to 89.16% compared to other serving configurations. Similarly, optimized runtime engines like ONNX with the CPU execution provider achieved from 8.98% up to 72.04% energy savings within CPU-based configurations. Also, TORCH paired with CUDA exhibited efficient computing-resource utilization. Serving configuration choice significantly impacts resource utilization. While further research is needed, we recommend the above configurations best suited to software engineers' requirements for enhancing serving resource utilization efficiency.

小模型能效优化推理加速代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。