提出新型离散状态模型Naju，实现长序列记忆的持久保留与主动更新兼顾。
Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory
- 将状态更新分解为遗忘门、写入增益和输入映射，独立控制记忆保留与写入。
- 在4倍训练长度下仍保持强保留与强写入能力，优于现有基线模型。
- 无需稳定性正则化，适合长序列建模任务，如语言生成与长程依赖学习。
长序列记忆追踪对循环状态提出两个对立要求：在长时域内近乎无损地保留存储绑定，同时主动覆盖过时信息。在我们的诊断测试中，最强的高效基线模型往往只能很好解决其中一方面。连续时间参数化状态空间模型（SSMs）如Mamba通过零阶保持离散化连续系统获得离散递归；我们认为这对记忆追踪而言是不必要的迂回，因此直接参数化离散转移。Naju（原生自适应连接单元）将递归更新分解为 $x_n = f_n\odot x_{n-1} + i_n\odot(B_n u_n)$，明确分离出学习的遗忘门 $f_n$（离散极点）、独立写入增益 $i_n$ 以及依赖输入的写入/读取映射。由于Sigmoid遗忘门满足 $0 < f_n < 1$，每个冻结局部坐标天然为Schur稳定，全时变递归在一致有界假设下满足衰减记忆/BIBO有界性，无需稳定性正则化。我们形式化了耦合设计的关键结构限制：任何非扩张互补单门递归将有效保留率 $r$ 与写入增益 $w$ 通过 $|r| + w \leq 1$ 紧密关联，导致高保留迫使弱写入；解耦 $f_n$ 与 $i_n$ 可消除此约束。实验表明，Naju是唯一在4倍训练长度下仍保持强保留与强写入能力的模型。除诊断套件外，我们在WikiText-103语言建模、Long Range Arena及多查询关联回忆任务上评估，结果一致显示其兼具强大长程记忆与竞争力或优越性能，主对比中超越Mamba基线，且与Transformer相当，同时保持线性时间与线性内存扩展。
原文摘要 · Abstract (English)
Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones. In our diagnostic suite, the strongest efficient baselines tend to solve only one side well. Continuous-time-parameterized state-space models (SSMs) such as Mamba obtain their discrete recurrence by zero-order-hold discretization of a continuous-time system; we argue that this detour is unnecessary for memory tracking and parameterize the discrete transition directly. Naju (Native Adaptive Junction Unit) factorizes the recurrent update, schematically $x_n = f_n\odot x_{n-1} + i_n\odot(B_n u_n)$, into an explicit discrete pole (a learned forget gate $f_n$), an independent write gain $i_n$, and input-dependent write/read maps. Since the sigmoid pole satisfies $0<f_n<1$, each frozen local coordinate is Schur-stable by construction, and the full time-varying recurrence satisfies a fading-memory/BIBO bound under uniform boundedness assumptions, with no stability regularizer. We formalize the key structural limitation of coupled designs: any non-expansive complementary single-gate recurrence ties the effective retention $r$ and write gain $w$ through $|r|+w\le 1$, so near-complete retention forces weak writing; decoupling $f_n$ from $i_n$ removes this constraint. Empirically, Naju is the only evaluated model that remains strong on both retention and overwriting at 4x the training length. Beyond the diagnostic suite, we evaluate Naju on WikiText-103 language modeling, Long Range Arena, and multi-query associative recall. Across these settings, Naju consistently combines strong long-range memory with competitive or superior performance, outperforming the Mamba baselines in the principal comparisons while remaining competitive with the Transformer and preserving linear-time, linear-memory scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。