提出高效在线学习线性系统的新算法,内存随系统复杂度自适应增长。
A Memory Efficient Unified Algorithm for Online Learning of Linear Dynamical Systems
- 统一算法处理各类线性动态系统,参数量仅与不稳定模式数k相关。
- 在高维系统上,同等参数下性能显著优于现有方法。
- 理论证明最小需k个滤波器,适用于需要低内存的实时控制场景。
为从观测中稳定未知线性动态系统(LDS)的挑战所驱动,本文研究在线预测这一基础前提。目标是在不依赖完整隐状态维度的前提下,实现次线性后悔率,并使内存占用适配动态系统的内在复杂度。聚焦于实际关键情形:系统具有低不稳定复杂度——即位于实稳定区间外但不快速衰减的特征值,以及非半单模态,可能嵌入于更高维度的稳定实谱中;用k表示此类模式数量。该情形是稳定化可行的核心:我们证明,高不稳定复杂度系统需指数级大控制量才能稳定。因此,预测对稳定化有意义的前提正是不稳定复杂度小。在此范围内,我们提出一种统一在线算法,可处理所有类型LDS(包括复数或爆炸模态的不可对角化系统),参数量为$ ilde{O}(k)$。最后,我们证明下界:任何基于滤波器的预测器至少需要k个滤波器。实验验证理论:在高维系统上,相同参数预算下,本方法性能明显超越已有方法。
原文摘要 · Abstract (English)
Motivated by the challenge of stabilizing a general unknown linear dynamical system (LDS) from observations, we study the natural prerequisite of online prediction. Our goal is to achieve sublinear regret with a memory footprint that adapts to the intrinsic complexity of the dynamics rather than the full hidden-state dimension. We focus on the practically central regime of systems with low instability complexity -- eigenvalues outside the real stable interval that do not decay rapidly, together with non-semisimple modes -- potentially embedded in an otherwise stable real spectrum of much higher dimension; we write $k$ for this count. This regime is the primary setting in which stabilization is plausible: we show that many systems with high instability complexity cannot be stabilized without exponentially large controls. Thus, prediction is meaningful for stabilization precisely when the instability complexity is small. Within this regime, we introduce a unified online algorithm that handles every LDS (including non-diagonalizable systems with complex or exploding modes) with a learnable parameter count of $\widetilde{O}(k)$. Finally, we prove a lower bound showing that $k$ is a valid complexity measure: any filter-based predictor needs at least $k$ filters. Experiments corroborate our theory: on a high-dimensional system, our predictor sharply outperforms prior methods at an equal parameter budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。