用正交多项式预处理序列，显著提升预测算法性能。
Universal Sequence Preconditioning
- 用切比雪夫/勒让德多项式系数卷积目标序列实现通用预处理
- 首次获得与隐藏维度无关的亚线性后悔界（对称/非对称系统均成立）
- 适用于RNN等多类算法，可推广至非线性系统
我们研究序列预测中的预处理问题。从线性动态系统的理论视角出发，发现对目标序列进行卷积相当于对隐藏转移矩阵施加一个多项式。基于此洞察，提出一种通用预处理方法：将目标序列与切比雪夫或勒让德等正交多项式的系数进行卷积。证明该方法可降低两类预测算法的后悔值，并首次在具有边缘表型和非对称转移矩阵的系统上，获得不依赖隐藏维度的亚线性后悔界（仅含对数因子）。大量合成及真实世界实验表明，这一简单预处理策略能有效提升多种算法（包括循环神经网络）性能，并可推广至线性动态系统以外的信号类型。
原文摘要 · Abstract (English)
We study the problem of preconditioning in sequential prediction. From the theoretical lens of linear dynamical systems, we show that convolving the target sequence corresponds to applying a polynomial to the hidden transition matrix. Building on this insight, we propose a universal preconditioning method that convolves the target with coefficients from orthogonal polynomials such as Chebyshev or Legendre. We prove that this approach reduces regret for two distinct prediction algorithms and yields the first ever sublinear and hidden-dimension-independent regret bounds (up to logarithmic factors) that hold for systems with marginally table and asymmetric transition matrices. Finally, extensive synthetic and real-world experiments show that this simple preconditioning strategy improves the performance of a diverse range of algorithms, including recurrent neural networks, and generalizes to signals beyond linear dynamical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。