提出分层去中心化控制框架,提升系统资源分配效率。
Hierarchical Decentralized Stochastic Control for Cyber-Physical Systems
- 两级控制器协同:全局慢控约束局部快控
- 有限时域框架让局部决策更自主,性能略低于无限时域
- 理论证明最优策略存在且算法收敛,适用于电网与灾情管理
本文提出一种面向网络物理系统(CPS)的双时标分层去中心化控制架构,包含一个全局控制器(GC)和N个本地控制器(LCs)。GC以较慢时标运行,对LCs的动作施加预算约束;而LCs以较快时标运作。该框架适用于能源网规划、野火管理等分布式资源分配场景。提出了两种优化范式:COpt中GC与LCs共同优化无限时域折扣奖励;FOpt中LCs优化有限时域周期奖励,而GC优化无限时域奖励。尽管奖励函数相同,因时域差异导致最优策略不同。其中,FOpt赋予LCs更大自主性,仅基于局部目标确定策略。我们建立了形式化框架,证明了最优策略的存在性及值迭代算法的收敛性,并推导出COpt始终优于FOpt的定量边界。最后,给出了使两者等价的充分结构条件。
原文摘要 · Abstract (English)
This paper introduces a two-timescale hierarchical decentralized control architecture for Cyber-Physical Systems (CPS). The system consists of a global controller (GC), and N local controllers (LCs). The GC operates at a slower timescale, imposing budget constraints on the actions of LCs, which function at a faster timescale. Applications can be found in energy grid planning, wildfire management, and other decentralized resource allocation problems. We propose and analyze two optimization frameworks for this setting: COpt and FOpt. In COpt, both GC and LCs together optimize infinite-horizon discounted rewards, while in FOpt the LCs optimize finite-horizon episodic rewards, and the GC optimizes infinite-horizon rewards. Although both frameworks share identical reward functions, their differing horizons can lead to different optimal policies. In particular, FOpt grants greater autonomy to LCs by allowing their policies to be determined only by local objectives, unlike COpt. To our knowledge, these frameworks have not been studied in the literature. We establish the formulations, prove the existence of optimal policies, and prove the convergence of their value iteration algorithms. We further show that COpt always achieves a higher value function than FOpt and derive explicit bounds on their difference. Finally, we establish a set of sufficient structural conditions under which the two frameworks become equivalent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。