提出稳定算法,让状态空间模型高效处理长序列依赖。
Fast convolution algorithm for state space models
- 用z域多项式近似传递函数,实现无条件稳定计算
- 仅需最多2L次矩阵乘法即可计算L个状态,效率提升显著
- 允许部分特征值大于1,支持更多结构化加速方法
我们提出一种在时域中应用线性时不变系统(LTI)矩阵传递函数的无条件稳定算法。状态空间模型(SSMs)中用于建模长程依赖的LTI系统状态矩阵特征值接近1,若其任一特征值模略大于1,标准递推将变得不稳定。这可能出现在用结构化矩阵近似原矩阵以将矩阵-向量乘法复杂度从$\mathcal{O}(m^2)$降至$\mathcal{O}(m)$或$\mathcal{O}(m\log m)$时。本文引入一种新算法,通过z域有理传递函数的矩阵多项式近似(阶数为$2^{N+1}-1$),其中N可调以达到用户指定精度。采用时域级联实现,计算L个状态最多只需$2L$次矩阵-向量乘法(标准递推需$L$次)。该算法无需保证近似状态矩阵所有特征值模严格小于1,在所需精度下允许部分特征值模超过1。因此,该方法可使用更广泛的结构化近似方式,降低计算成本,文中简要介绍了若干可用于此目的的结构化近似方法。
原文摘要 · Abstract (English)
We present an unconditionally stable algorithm for applying matrix transfer function of a linear time invariant system (LTI) in time domain. The state matrix of an LTI system used for modeling long range dependencies in state space models (SSMs) has eigenvalues close to $1$. The standard recursion defining LTI system becomes unstable if the $m\times m$ state matrix has just one eigenvalue with absolute value even slightly greater than 1. This may occur when approximating a state matrix by a structured matrix to reduce the cost of matrix-vector multiplication from $\mathcal{O}\left(m^{2}\right)$ to $\mathcal{O}\left(m\right)$ or $\mathcal{O}\left(m\log m\right).$ We introduce an unconditionally stable algorithm that uses an approximation of the rational transfer function in the z-domain by a matrix polynomial of degree $2^{N+1}-1$, where $N$ is chosen to achieve any user-selected accuracy. Using a cascade implementation in time domain, applying such transfer function to compute $L$ states requires no more than $2L$ matrix-vector multiplications (whereas the standard recursion requires $L$ matrix-vector multiplications). However, using unconditionally stable algorithm, it is not necessary to assure that an approximate state matrix has all eigenvalues with absolute values strictly less than 1 i.e., within the desired accuracy, the absolute value of some eigenvalues may possibly exceed $1$. Consequently, this algorithm allows one to use a wider variety of structured approximations to reduce the cost of matrix-vector multiplication and we briefly describe several of them to be used for this purpose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。