实测验证代码碳排放工具的准确性,发现误差高达40%。
Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements
- 通过真实能耗测量对比评估静态与动态估算方法
- 发现现有工具误差最高达40%,普遍高估或低估能耗
- 提供可复用代码与指南,推动可持续AI研究
尽管机器学习和人工智能为创新带来机遇,但其快速发展也显著影响环境。为此,开发了如ML Emissions Calculator和CodeCarbon等量化工具,用于估算运行AI模型的能耗与碳排放。这些工具易于集成到项目中,但依赖简化假设并忽略关键因素,引发对估算准确性的质疑。本研究通过数百次AI实验的实地测量,系统评估了静态与动态能耗估算方法的可靠性。基于提出的验证框架,揭示了AI能耗需求及估算误差的深层机制。结果显示,现有估算方法虽总体符合能耗趋势,但存在高达40%的系统性偏差。研究提供了能源估算质量的实证证据,验证了主流工具的可信度,并提出改进路线,同时开源代码以支持其他工具与领域的扩展验证,为资源感知型机器学习与人工智能可持续性研究做出重要贡献。
原文摘要 · Abstract (English)
Although machine learning (ML) and artificial intelligence (AI) present fascinating opportunities for innovation, their rapid development is also significantly impacting our environment. In response to growing resource-awareness in the field, quantification tools such as the ML Emissions Calculator and CodeCarbon were developed to estimate the energy consumption and carbon emissions of running AI models. They are easy to incorporate into AI projects, however also make pragmatic assumptions and neglect important factors, raising the question of estimation accuracy. This study systematically evaluates the reliability of static and dynamic energy estimation approaches through comparisons with ground-truth measurements across hundreds of AI experiments. Based on the proposed validation framework, investigative insights into AI energy demand and estimation inaccuracies are provided. While generally following the patterns of AI energy consumption, the established estimation approaches are shown to consistently make errors of up to 40%. By providing empirical evidence on energy estimation quality and errors, this study establishes transparency and validates widely used tools for sustainable AI development. It moreover formulates guidelines for improving the state-of-the-art and offers code for extending the validation to other domains and tools, thus making important contributions to resource-aware ML and AI sustainability research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。