用小波构造长序列建模新方法，突破传统基函数限制。
WaLRUS: Wavelets for Long-range Representation Using SSMs
- 基于小波框架构建状态空间模型，支持非正交冗余基
- 使用达布奇斯小波实现更灵活的长程依赖建模
- 适合需要高效处理长序列数据的研究者
状态空间模型（SSMs）在建模序列数据的长程依赖方面表现出色。尽管最近的HiPPO方法表现优异，并成为S4和Mamba等模型的基础，但仍受限于对少数特定、良好行为基函数的闭式解依赖。SaFARi框架通过允许从任意帧构造SSM，包括非正交和冗余的基，扩展了这一方法，从而实现了SSM家族中无限多样的“物种”。本文提出WaLRUS（Wavelets for Long-range Representation Using SSMs），一种基于达布奇斯小波实现的SaFARi新实例。
原文摘要 · Abstract (English)
State-Space Models (SSMs) have proven to be powerful tools for modeling long-range dependencies in sequential data. While the recent method known as HiPPO has demonstrated strong performance, and formed the basis for machine learning models S4 and Mamba, it remains limited by its reliance on closed-form solutions for a few specific, well-behaved bases. The SaFARi framework generalized this approach, enabling the construction of SSMs from arbitrary frames, including non-orthogonal and redundant ones, thus allowing an infinite diversity of possible "species" within the SSM family. In this paper, we introduce WaLRUS (Wavelets for Long-range Representation Using SSMs), a new implementation of SaFARi built from Daubechies wavelets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。