无需梯度计算的元强化学习框架,快速适应电网故障恢复新场景。
Toward Adaptive Grid Resilience: A Gradient-Free Meta-RL Framework for Critical Load Restoration
- 结合一阶元学习与进化策略,实现无梯度可扩展策略搜索。
- 在多个测试系统中优于传统RL与模型预测控制,恢复效率更高。
- 适合高比例可再生能源电网的实时负载恢复,尤其适用于未知故障场景。
极端事件后恢复关键负荷需自适应控制以维持配电网络韧性,但可再生能源出力不确定性、可调度资源有限及非线性动态使有效恢复困难。强化学习可在不确定性下优化序列决策,但标准RL泛化能力差,且对新故障配置或发电模式需大量重训练。本文提出一种元引导的无梯度强化学习(MGF-RL)框架,从历史故障经验中学习可迁移的初始化,仅需少量任务特定调优即可快速适应未见场景。MGF-RL融合一阶元学习与进化策略,在无需梯度计算的前提下支持非线性、有约束的配电系统动态。在IEEE 13-bus和IEEE 123-bus测试系统上,MGF-RL在可靠性、恢复速度与适应效率方面均优于标准RL、基于MAML的元强化学习及模型预测控制,且能泛化至未见故障与可再生能源模式,所需微调轮次远少于传统方法。此外,本文给出子线性后悔界,将适应效率与任务相似性及环境变化相关联,支撑实证性能,推动该框架在高可再生能源配电系统中的实时应用。
原文摘要 · Abstract (English)
Restoring critical loads after extreme events demands adaptive control to maintain distribution-grid resilience, yet uncertainty in renewable generation, limited dispatchable resources, and nonlinear dynamics make effective restoration difficult. Reinforcement learning (RL) can optimize sequential decisions under uncertainty, but standard RL often generalizes poorly and requires extensive retraining for new outage configurations or generation patterns. We propose a meta-guided gradient-free RL (MGF-RL) framework that learns a transferable initialization from historical outage experiences and rapidly adapts to unseen scenarios with minimal task-specific tuning. MGF-RL couples first-order meta-learning with evolutionary strategies, enabling scalable policy search without gradient computation while accommodating nonlinear, constrained distribution-system dynamics. Experiments on IEEE 13-bus and IEEE 123-bus test systems show that MGF-RL outperforms standard RL, MAML-based meta-RL, and model predictive control across reliability, restoration speed, and adaptation efficiency under renewable forecast errors. MGF-RL generalizes to unseen outages and renewable patterns while requiring substantially fewer fine-tuning episodes than conventional RL. We also provide sublinear regret bounds that relate adaptation efficiency to task similarity and environmental variation, supporting the empirical gains and motivating MGF-RL for real-time load restoration in renewable-rich distribution grids.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。