对比7种强化学习算法在雅达利游戏上的能耗与碳排放，揭示节能潜力。
Greener Deep Reinforcement Learning: Analysis of Energy and Carbon Efficiency Across Atari Benchmarks
- 在10个雅达利游戏中测试7种主流DRL算法的实时能耗。
- 最优算法比最耗能者节能24%、减排68%、省成本68%。
- 为绿色强化学习提供可量化的算法选型参考。
深度强化学习（DRL）日益增长的计算需求引发了对其环境与经济成本的担忧。尽管学习性能的算法效率已广泛研究，但其能耗、温室气体排放和资金成本仍缺乏系统评估。本文对七种先进DRL算法——DQN、TRPO、A2C、ARS、PPO、RecurrentPPO 和 QR-DQN——在 Stable Baselines 中实现，每个算法在10个 Atari 2600 游戏上训练一百万步，并实时测量功耗，估算总能量消耗、二氧化碳当量排放及基于美国全国平均电价的电费。结果显示，各算法间存在显著差异：部分算法在性能相近时，能耗降低高达24%（ARS vs. DQN），CO2 排放减少近68%，成本下降约68%（QR-DQN vs. RecurrentPPO）。我们进一步分析了学习性能、训练时间、能耗与财务成本之间的权衡，指出算法选择可在不牺牲性能的前提下显著降低环境与经济影响。本研究为发展能源敏感、成本高效的DRL实践提供了可行建议，并为未来算法设计与评估中融入可持续性考量奠定基础。
原文摘要 · Abstract (English)
The growing computational demands of deep reinforcement learning (DRL) have raised concerns about the environmental and economic costs of training large-scale models. While algorithmic efficiency in terms of learning performance has been extensively studied, the energy requirements, greenhouse gas emissions, and monetary costs of DRL algorithms remain largely unexplored. In this work, we present a systematic benchmarking study of the energy consumption of seven state-of-the-art DRL algorithms, namely DQN, TRPO, A2C, ARS, PPO, RecurrentPPO, and QR-DQN, implemented using Stable Baselines. Each algorithm was trained for one million steps each on ten Atari 2600 games, and power consumption was measured in real-time to estimate total energy usage, CO2-Equivalent emissions, and electricity cost based on the U.S. national average electricity price. Our results reveal substantial variation in energy efficiency and training cost across algorithms, with some achieving comparable performance while consuming up to 24% less energy (ARS vs. DQN), emitting nearly 68% less CO2, and incurring almost 68% lower monetary cost (QR-DQN vs. RecurrentPPO) than less efficient counterparts. We further analyze the trade-offs between learning performance, training time, energy use, and financial cost, highlighting cases where algorithmic choices can mitigate environmental and economic impact without sacrificing learning performance. This study provides actionable insights for developing energy-aware and cost-efficient DRL practices and establishes a foundation for incorporating sustainability considerations into future algorithmic design and evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。