让LSTM学会自我反思，大幅提升长序列记忆稳定性。
EchoLSTM: A Self-Reflective Recurrent Network for Stabilizing Long-Range Memory
- 通过输出条件门控实现自省式记忆调节
- 在干扰任务中准确率达69.0%，比标准LSTM高33个百分点
- 参数效率超5倍于Transformer，适合资源受限场景
标准循环神经网络（如LSTM）难以建模长程依赖，尤其在存在噪声或误导信息的序列中表现不佳。本文提出输出条件门控机制，使模型能根据自身历史推断动态调节内部记忆门，形成稳定记忆的反馈回路。最终模型EchoLSTM结合该机制与注意力结构。在自定义的干扰信号任务中，其准确率达到69.0%，较标准LSTM基线提升33个百分点。在标准ListOps基准上，性能达69.8%，接近现代Transformer模型的71.8%，且参数量不足其1/5。触发敏感性测试进一步表明，该自省机制显著增强了记忆系统的鲁棒性。
原文摘要 · Abstract (English)
Standard Recurrent Neural Networks, including LSTMs, struggle to model long-range dependencies, particularly in sequences containing noisy or misleading information. We propose a new architectural principle, Output-Conditioned Gating, which enables a model to perform self-reflection by modulating its internal memory gates based on its own past inferences. This creates a stabilizing feedback loop that enhances memory retention. Our final model, the EchoLSTM, integrates this principle with an attention mechanism. We evaluate the EchoLSTM on a series of challenging benchmarks. On a custom-designed Distractor Signal Task, the EchoLSTM achieves 69.0% accuracy, decisively outperforming a standard LSTM baseline by 33 percentage points. Furthermore, on the standard ListOps benchmark, the EchoLSTM achieves performance competitive with a modern Transformer model, 69.8% vs. 71.8%, while being over 5 times more parameter-efficient. A final Trigger Sensitivity Test provides qualitative evidence that our model's self-reflective mechanism leads to a fundamentally more robust memory system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。