通过延迟微分方程增强状态空间模型的记忆能力，提升长序列精准检索性能。
DSSMs: State Space Models with Explicit Memory via Delay Differential Equations
- 引入延迟反馈机制，显式建模历史信息以突破固定状态容量限制。
- 在标准序列任务上优于S4D，在延迟检索任务上显著提升性能。
- 适用于需要长程记忆与精确上下文召回的场景，如日志分析、金融预测。
状态空间模型（SSMs）已成为高效长序列建模的强大范式，支持并行训练和线性时间递归推理。然而，与其他递归架构类似，SSMs必须将无限历史压缩为固定大小的状态，限制了上下文保留能力，导致长程上下文的精确检索本质上困难。为此，我们提出延迟状态空间模型（DSSMs），一种受延迟微分方程（DDE）启发的对角SSM扩展，通过显式延迟状态反馈增强离散SSM的递推关系。实现显式延迟反馈需解决稳定性参数化、历史管理及FFT训练等挑战。我们基于简单的无延迟依赖稳定性条件，提出实用的离散化与参数化方法。为避免直接进行时域核构造，我们推导出DSSM的传递函数，并在频域中计算核函数，利用核轮廓偏移抑制混叠，恢复准确的FFT训练。实验表明，DSSMs在目标延迟检索任务上显著提升性能，多数标准序列指标优于S4D，其余接近其表现。
原文摘要 · Abstract (English)
State Space Models (SSMs) have emerged as a powerful paradigm for efficient long-sequence modeling, offering parallel training and fast linear-time recurrent inference. However, like other recurrent architectures, SSMs must compress an unbounded history into a fixed-size state, which limits context retention and makes precise retrieval over long-range context inherently difficult. To overcome this limitation, we propose Delay State Space Models (DSSMs), a delay differential equation (DDE)-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback. Making explicit delayed feedback practical requires new stability parameterization, history management, and FFT-training tools. We address these challenges with a practical discretization and parameterization grounded in a simple delay-independent stability condition. To bypass direct time-domain kernel construction, we derive the DSSM transfer function and compute kernels in the frequency domain, using a kernel contour shift to suppress aliasing and recover accurate FFT training. Empirically, DSSMs substantially improve targeted delayed-retrieval tasks while outperforming S4D on most standard sequence metrics and remaining close on the others.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。