arXiv:2412.02602cs.CLcs.AI2024-12被引 1

提出碳效比指标,评估小模型在性能与碳排放间的平衡。

CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs

  • 引入新指标CEGI,衡量每百万参数每单位性能提升的碳排放
  • 微调小模型可逼近大模型性能,碳排放显著更低
  • 低比特量化能提能效且不降性能,适合绿色AI开发

本文分析了小型语言模型(SLMs)和视觉语言模型(VLMs)在图像描述、视觉问答(VQA)、对话摘要和文本转SQL四个核心任务上的表现,评估其性能与碳排放之间的权衡。选取基于Qwen和LLaMA架构的多种模型,涵盖参数量、量化等级和微调参数等变体,计算各模型的性能与碳排放。为量化该权衡关系,提出新型指标CEGI(碳效比增益指数),表示每百万可训练参数每单位性能提升所对应的碳排放量。实验表明,微调SLMs和VLMs可在性能上接近大型语言模型(LLMs)的同时,实现显著更低的碳排放;更大的模型带来的边际精度提升不足以抵消其碳排放的急剧上升。采用更低比特量化进一步提升了能效,且不影响性能。研究强调高性能与环境可持续性的平衡,为绿色AI发展提供关键评估工具。

原文摘要 · Abstract (English)

This paper analyzes the performance of Small Language Models (SLMs) and Vision Language Models (VLMs) and evaluates the trade-off between model performance and carbon emissions across 4 essential tasks: Image Captioning, Visual Question Answering (VQA), Dialogue Summarization and Text-to-SQL conversion. Various SLMs and VLMs belonging to the Qwen and LLaMA architecture family are chosen and variants based on model size in terms of the number of parameters, quantization level and fine-tuning parameters are evaluated. The model variant's performance and carbon emissions are calculated. To quantify the trade-off between model performance and carbon emissions, we introduce a novel metric called CEGI (Carbon Efficient Gain Index). This metric represents the carbon emission per unit percentage gain per million trainable parameters . This metric provides a normalized measure to compare model's efficiency in terms of performance improvement relative to their environmental cost. The experiment's outcome demonstrates that fine-tuning SLMs and VLMs can achieve performance levels comparable to Large Language Models (LLMs) while producing significantly less carbon emissions. Our findings suggest that the marginal gains in accuracy from larger models do not justify the substantial increase in carbon emissions. Leveraging lower-bit quantization levels, the proposed metric further enhances energy efficiency without compromising performance. This study highlights balancing high performance and environmental sustainability. It offers a valuable metric for selecting models suitable for environmentally-friendly AI development.

模型效率碳排放小模型绿色AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。