通过系统级指标增强智能体观测与奖励,实现去中心化能源市场中的隐性协作。
MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
- 用系统级指标扩展智能体的观测与奖励,促进自主学习协同策略。
- 在不同储能配置下验证了市场效率提升与电网稳定性增强。
- 开源可复现,适合研究未来分布式能源系统的协同机制。
本文提出一个面向去中心化本地能源市场中隐性协作的新型开放源代码多智能体强化学习仿真框架,将市场建模为部分可观测马尔可夫决策过程,并实现为MARL兼容的Gymnasium环境。该框架包含模块化市场平台(支持即插即用的清算机制)、物理约束的智能体模型(含电池储能)、真实电网拓扑及全面分析工具,用于评估涌现的协调行为。核心贡献是一种新方法:通过在智能体的观测与奖励中引入系统级关键绩效指标,使其能够独立学习有利于整个系统的策略,实现集体有益结果而无需显式通信。通过代表性案例研究(代码见https://github.com/salazarna/marlem),验证了不同储能部署对系统性能的影响,展示了框架在促进涌现协调、提升市场效率和增强电网稳定性方面的潜力。该仿真工具具有灵活性、可扩展性和可复现性,适用于研究人员与从业者设计、测试与验证未来智能分布式能源系统的策略。
原文摘要 · Abstract (English)
This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs, modeled as a decentralized partially observable Markov decision process and implemented as a Gymnasium environment for MARL. Our framework features a modular market platform with plug-and-play clearing mechanisms, physically constrained agent models (including battery storage), a realistic grid network, and a comprehensive analytics suite to evaluate emergent coordination. The main contribution is a novel method to foster implicit cooperation, where agents' observations and rewards are enhanced with system-level key performance indicators to enable them to independently learn strategies that benefit the entire system and aim for collectively beneficial outcomes without explicit communication. Through representative case studies (available in a dedicated GitHub repository in https://github.com/salazarna/marlem, we show the framework's ability to analyze how different market configurations (such as varying storage deployment) impact system performance. This illustrates its potential to facilitate emergent coordination, improve market efficiency, and strengthen grid stability. The proposed simulation framework is a flexible, extensible, and reproducible tool for researchers and practitioners to design, test, and validate strategies for future intelligent, decentralized energy systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。