arXiv:2411.17729math.NAcs.AI2024-11被引 1

提出稳定算法,让状态空间模型高效处理长序列依赖。

Fast convolution algorithm for state space models

  • 用z域多项式近似传递函数,实现无条件稳定计算
  • 仅需最多2L次矩阵乘法即可计算L个状态,效率提升显著
  • 允许部分特征值大于1,支持更多结构化加速方法

我们提出一种在时域中应用线性时不变系统(LTI)矩阵传递函数的无条件稳定算法。状态空间模型(SSMs)中用于建模长程依赖的LTI系统状态矩阵特征值接近1,若其任一特征值模略大于1,标准递推将变得不稳定。这可能出现在用结构化矩阵近似原矩阵以将矩阵-向量乘法复杂度从$\mathcal{O}(m^2)$降至$\mathcal{O}(m)$或$\mathcal{O}(m\log m)$时。本文引入一种新算法,通过z域有理传递函数的矩阵多项式近似(阶数为$2^{N+1}-1$),其中N可调以达到用户指定精度。采用时域级联实现,计算L个状态最多只需$2L$次矩阵-向量乘法(标准递推需$L$次)。该算法无需保证近似状态矩阵所有特征值模严格小于1,在所需精度下允许部分特征值模超过1。因此,该方法可使用更广泛的结构化近似方式,降低计算成本,文中简要介绍了若干可用于此目的的结构化近似方法。

原文摘要 · Abstract (English)

We present an unconditionally stable algorithm for applying matrix transfer function of a linear time invariant system (LTI) in time domain. The state matrix of an LTI system used for modeling long range dependencies in state space models (SSMs) has eigenvalues close to $1$. The standard recursion defining LTI system becomes unstable if the $m\times m$ state matrix has just one eigenvalue with absolute value even slightly greater than 1. This may occur when approximating a state matrix by a structured matrix to reduce the cost of matrix-vector multiplication from $\mathcal{O}\left(m^{2}\right)$ to $\mathcal{O}\left(m\right)$ or $\mathcal{O}\left(m\log m\right).$ We introduce an unconditionally stable algorithm that uses an approximation of the rational transfer function in the z-domain by a matrix polynomial of degree $2^{N+1}-1$, where $N$ is chosen to achieve any user-selected accuracy. Using a cascade implementation in time domain, applying such transfer function to compute $L$ states requires no more than $2L$ matrix-vector multiplications (whereas the standard recursion requires $L$ matrix-vector multiplications). However, using unconditionally stable algorithm, it is not necessary to assure that an approximate state matrix has all eigenvalues with absolute values strictly less than 1 i.e., within the desired accuracy, the absolute value of some eigenvalues may possibly exceed $1$. Consequently, this algorithm allows one to use a wider variety of structured approximations to reduce the cost of matrix-vector multiplication and we briefly describe several of them to be used for this purpose.

状态空间模型高效计算稳定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。