arXiv:2602.16062eess.SYcs.CE2026-02

用隐式信号实现去中心化电网协同,省通信、保稳定。

Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets

  • 通过系统级指标间接感知全局状态,实现无直接通信的协同决策
  • 最优配置达91.7%协调度,且去中心化方案降低31%电网波动
  • 适合关注电网稳定与隐私保护的能源系统研究者

本文提出隐式协作框架,使去中心化代理在无需直接点对点通信的情况下逼近本地能源市场的最优协调。将问题建模为部分可观测马尔可夫决策过程,采用多智能体强化学习,利用系统层面的绩效指标(即栖息信号)推断并响应全局状态。在IEEE 34节点拓扑上,通过3×3因子设计评估三种训练范式(CTCE、CTDE、DTDE)与三种算法(PPO、APPO、SAC)。结果表明,APPO-DTDE为最优配置,协调度达理论中心化基准(CTCE)的91.7%。然而效率与稳定性存在权衡:中心化方案虽具最高分配效率(点对点交易比0.6),但全去中心化(DTDE)在物理稳定性上更优,其电网平衡方差较混合架构降低31%,形成可预测的进口依赖型负荷模式,简化电网调控。拓扑分析揭示代理自发形成稳定交易社区以降低阻塞惩罚。尽管SAC在混合设置中表现优异,但在去中心化环境因熵驱动不稳定性而失效。研究证明,栖息信号足以支撑复杂电网协调,提供一种鲁棒、隐私友好的替代中心化通信基础设施的方案。

原文摘要 · Abstract (English)

This paper proposes implicit cooperation, a framework enabling decentralized agents to approximate optimal coordination in local energy markets without explicit peer-to-peer communication. We formulate the problem as a decentralized partially observable Markov decision problem that is solved through a multi-agent reinforcement learning task in which agents use stigmergic signals (key performance indicators at the system level) to infer and react to global states. Through a 3x3 factorial design on an IEEE 34-node topology, we evaluated three training paradigms (CTCE, CTDE, DTDE) and three algorithms (PPO, APPO, SAC). Results identify APPO-DTDE as the optimal configuration, achieving a coordination score of 91.7% relative to the theoretical centralized benchmark (CTCE). However, a critical trade-off emerges between efficiency and stability: while the centralized benchmark maximizes allocative efficiency with a peer-to-peer trade ratio of 0.6, the fully decentralized approach (DTDE) demonstrates superior physical stability. Specifically, DTDE reduces the variance of grid balance by 31% compared to hybrid architectures, establishing a highly predictable, import-biased load profile that simplifies grid regulation. Furthermore, topological analysis reveals emergent spatial clustering, where decentralized agents self-organize into stable trading communities to minimize congestion penalties. While SAC excelled in hybrid settings, it failed in decentralized environments due to entropy-driven instability. This research proves that stigmergic signaling provides sufficient context for complex grid coordination, offering a robust, privacy-preserving alternative to expensive centralized communication infrastructure.

多智能体能源市场强化学习去中心化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。