用二阶方法预处理序列，让长记忆系统预测更准且不随维度变差。
The Power of Second Order Methods for Sequence Preconditioning
- 用二阶算法学习短自回归模型，压缩记忆信息。
- 在复平面除负实轴附近扇形区外的系统上，误差增长为δ⁻⁴log²T。
- 适合研究长时序预测与鲁棒控制的学者参考。
针对具有长记忆的边际稳定线性动态系统，传统序列预测方法的遗憾（regret）随隐状态维度线性增长。本文表明，仅使用二阶Vovk-Azoury-Warmuth（VAW）算法来学习一个短自回归输入模型（ARX），即可取得惊人效果：对于从谱位于复单位圆盘内但排除负实轴附近宽度为δ的扇形区域的边际稳定系统生成的有界序列数据，该算法实现维度无关的遗憾界 $O(δ^{-4} /log^2 T)$，目前为最优结果。关键贡献包括：1）利用“通用序列预处理”（USP）理论证明了最优自回归系数的存在性；2）应用VAW算法更高效地利用了USP带来的记忆压缩；3）通过分析圆扇形区域上的Faber多项式，将结果扩展至具有复数谱的系统。
原文摘要 · Abstract (English)
Sequence prediction methods for linear dynamical systems with long memory, i.e. marginally stable systems, typically achieve regret that grows linearly with the hidden dimension of the underlying generative model. While many methods have been developed to address this regime with varying success, we show that simply using the second-order Vovk-Azoury-Warmuth (VAW) algorithm to learn a short autoregressive-with-inputs (ARX) model achieves astoundingly strong results: for bounded sequential data from a marginally-stable linear dynamical system with spectra in the complex disk except for angular wedge of width $δ$ around the negative real axis, this algorithm achieves dimension-free regret $O\left( δ^{-4} \log^2 T \right)$. These bounds are state-of-the-art to our knowledge. The key components for our result come from 1) using the theory of ``Universal Sequence Preconditioning'' (USP) \cite{marsdenuniversal} to prove the existence of an optimal setting of autoregressive coefficients, 2) the application of VAW which takes better advantage of the memory compression provided by USP, and 3) the analysis of Faber polynomials on circular sectors to extend these results to systems with complex spectra.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。