首次量化生成式AI研究全周期算力与环境成本,揭示模型研发隐性能耗。
Environmental Footprint of GenAI Research: Insights from the Moshi Foundation Model
- 拆解模型研发各阶段算力消耗,涵盖实验、调试和失败训练
- 计算出70亿参数模型全程耗电超12万千瓦时,碳排放约85吨
- 为开源科研团队提供可落地的绿色AI研究优化建议
新多模态大语言模型(MLLM)持续快速迭代,其研发与部署正推动数据中心建设与硬件制造带来的能源消耗、温室气体排放及其他环境影响不断上升。由于主要研究机构缺乏透明度,生成式AI的环境影响常被简化为最终训练阶段的碳足迹,忽略了研发过程中的早期探索、反复试验、失败训练及调试等环节。本研究以70亿参数的实时对话语音-文本基础模型Moshi为例,由知名私营开放科学实验室Kyutai开发,通过细粒度分析其研发全过程的算力投入,首次系统量化了模型组件构建、训练阶段、早期实验、失败训练、调试与消融研究等环节的GPU时间。同时,采用生命周期评估方法,全面测算从硬件生产到使用阶段的能源与水资源消耗、温室气体排放及矿产资源枯竭情况。结果表明,整个研发周期累计耗电超过12万千瓦时,碳排放达约85吨。本研究提出可操作的降低算力与环境影响的策略,为可持续人工智能研究提供路径指引。
原文摘要 · Abstract (English)
New multi-modal large language models (MLLMs) are continuously being trained and deployed, following rapid development cycles. This generative AI frenzy is driving steady increases in energy consumption, greenhouse gas emissions, and a plethora of other environmental impacts linked to datacenter construction and hardware manufacturing. Mitigating the environmental consequences of GenAI remains challenging due to an overall lack of transparency by the main actors in the field. Even when the environmental impacts of specific models are mentioned, they are typically restricted to the carbon footprint of the final training run, omitting the research and development stages. In this work, we explore the impact of GenAI research through a fine-grained analysis of the compute spent to create Moshi, a 7B-parameter speech-text foundation model for real-time dialogue developed by Kyutai, a leading privately funded open science AI lab. For the first time, our study dives into the anatomy of compute-intensive MLLM research, quantifying the GPU-time invested in specific model components and training phases, as well as early experimental stages, failed training runs, debugging, and ablation studies. Additionally, we assess the environmental impacts of creating Moshi from beginning to end using a life cycle assessment methodology: we quantify energy and water consumption, greenhouse gas emissions, and mineral resource depletion associated with the production and use of datacenter hardware. Our detailed analysis allows us to provide actionable guidelines to reduce compute usage and environmental impacts of MLLM research, paving the way for more sustainable AI research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。