提出无需重置的RNN持续推理框架，提升长序列处理能力。
Never Reset Again: A Mathematical Framework for Continual Inference in Recurrent Neural Networks
- 设计自适应损失函数，动态调节梯度以区分有效信息与噪声。
- 在长序列任务中性能超越传统重置方法，保持高精度稳定输出。
- 适合实时流数据处理场景，降低计算开销与同步要求。
循环神经网络（RNN）广泛用于序列处理，但持续推理时因隐藏状态饱和需频繁重置，导致输入边界同步要求高且计算成本增加。为此，本文提出一种自适应损失函数，通过结合交叉熵与KL散度，根据输入信息量动态调节梯度，使网络能区分有效数据与噪声，实现无重置的持续推理。实验表明，该方法在多种RNN架构上均优于传统重置策略，尤其在持续任务中表现更佳，显著提升了RNN在流式应用中的理论与实践能力。
原文摘要 · Abstract (English)
Recurrent Neural Networks (RNNs) are widely used for sequential processing but face fundamental limitations with continual inference due to state saturation, requiring disruptive hidden state resets. However, reset-based methods impose synchronization requirements with input boundaries and increase computational costs at inference. To address this, we propose an adaptive loss function that eliminates the need for resets during inference while preserving high accuracy over extended sequences. By combining cross-entropy and Kullback-Leibler divergence, the loss dynamically modulates the gradient based on input informativeness, allowing the network to differentiate meaningful data from noise and maintain stable representations over time. Experimental results demonstrate that our reset-free approach outperforms traditional reset-based methods when applied to a variety of RNNs, particularly in continual tasks, enhancing both the theoretical and practical capabilities of RNNs for streaming applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。