在观测不全的群体动态系统中,实现最优在线控制。
Population Dynamics Control with Partial Observations
- 用假设恒定控制下的虚拟观测构造无记忆信号
- 设计新控制器参数化,实现近似最优 $ ilde{O}( ext{sqrt}(T))$ 误差
- 适合研究在线控制与概率单纯形约束问题的研究者
我们研究群体动态系统的控制问题,这是一类在概率单纯形上演化的线性动力系统,从在线非随机控制的角度出发。尽管Golowich等(2024)分析了完全可观测情形,但本文聚焦更现实的局部可观测情况,即仅能访问状态的低维表示。经典非随机控制中,输入为过去扰动的线性组合;但在部分观测下,扰动无法直接计算。Simchowitz等(2020)提出用无记忆信号(即零控制下的反事实观测)作为替代。这在单纯形约束下引发多重挑战:(1) 如何在零控制不可行的单纯形域内构造无记忆信号;(2) 如何设计足够表达力的凸控制器参数化以适配这些信号;(3) 如何在投影破坏代价函数凸性时维持单纯形约束。我们的主要贡献是提出一种新控制器,可实现对自然混合线性动态控制器类别的最优 $ ilde{O}( ext{sqrt}(T))$ 冗余。为解决上述挑战,我们基于适应单纯形域的恒定控制构造假设观测信号,引入新控制器参数化以逼近依赖于非无记忆观测的一般控制策略,并采用受Lattimore(2024)启发的新型凸扩展代理损失,规避投影引起的凸性问题。
原文摘要 · Abstract (English)
We study the problem of controlling population dynamics, a class of linear dynamical systems evolving on the probability simplex, from the perspective of online non-stochastic control. While Golowich et.al. 2024 analyzed the fully observable setting, we focus on the more realistic, partially observable case, where only a low-dimensional representation of the state is accessible. In classical non-stochastic control, inputs are set as linear combinations of past disturbances. However, under partial observations, disturbances cannot be directly computed. To address this, Simchowitz et.al. 2020 proposed to construct oblivious signals, which are counterfactual observations with zero control, as a substitute. This raises several challenges in our setting: (1) how to construct oblivious signals under simplex constraints, where zero control is infeasible; (2) how to design a sufficiently expressive convex controller parameterization tailored to these signals; and (3) how to enforce the simplex constraint on control when projections may break the convexity of cost functions. Our main contribution is a new controller that achieves the optimal $\tilde{O}(\sqrt{T})$ regret with respect to a natural class of mixing linear dynamic controllers. To tackle these challenges, we construct signals based on hypothetical observations under a constant control adapted to the simplex domain, and introduce a new controller parameterization that approximates general control policies linear in non-oblivious observations. Furthermore, we employ a novel convex extension surrogate loss, inspired by Lattimore 2024, to bypass the projection-induced convexity issue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。