ML训练环境代价持续上升,优化反导致更多能耗。
The Environmental Impacts of Machine Learning Training Keep Rising Evidencing Rebound Effect
- 分析10年主流AI模型训练全生命周期碳排放
- 硬件生产与训练能耗均指数级增长,即使优化也难抵消
- 提醒:提升效率不等于可持续,需控制训练规模
近年来机器学习方法在基准测试中表现提升,但计算需求不断攀升。尽管已提出硬件、算法和碳优化策略以减少能耗与环境影响,这些措施能否实现可持续的模型训练?本文估算过去十年中代表性人工智能系统(包括大语言模型)训练相关的环境影响,重点关注显卡的全生命周期。分析揭示两个关键趋势:第一,显卡生产阶段的环境影响持续上升;第二,即便考虑将训练迁移至碳强度较低地区,模型训练的能源消耗与环境影响仍呈指数增长。优化策略无法缓解训练带来的影响,表明存在反弹效应。研究显示,必须将硬件影响纳入全生命周期评估,而非仅关注使用阶段,否则会导致影响转移。结果表明,单纯提升效率无法确保机器学习的可持续性。降低人工智能的环境影响还需减少相关活动,并反思资源密集型训练的规模与频率。
原文摘要 · Abstract (English)
Recent Machine Learning (ML) approaches have shown increased performance on benchmarks but at the cost of escalating computational demands. Hardware, algorithmic and carbon optimizations have been proposed to curb energy consumption and environmental impacts. Can these strategies lead to sustainable ML model training? Here, we estimate the environmental impacts associated with training notable AI systems over the last decade, including Large Language Models, with a focus on the life cycle of graphics cards. Our analysis reveals two critical trends: First, the impacts of graphics cards production have increased steadily over this period; Second, energy consumption and environmental impacts associated with training ML models have increased exponentially, even when considering reduction strategies such as location shifting to places with less carbon intensive electricity mixes. Optimization strategies do not mitigate the impacts induced by model training, evidencing rebound effect. We show that the impacts of hardware must be considered over the entire life cycle rather than the sole use phase in order to avoid impact shifting. Our study demonstrates that increasing efficiency alone cannot ensure sustainability in ML. Mitigating the environmental impact of AI also requires reducing AI activities and questioning the scale and frequency of resource-intensive training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。