通过持续记忆训练，让导航模型记住更长时间的交互信息。
StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigation
- 用状态保持机制在训练中延续记忆，不每批重置
- 在长序列任务中表现显著优于传统模型
- 适合需要长期记忆的智能导航系统研究
有效的导航智能依赖于长期记忆以支持即时泛化和持续适应。然而，现有方法面临两难：模块化系统依赖显式地图但缺乏灵活性，而基于Transformer的端到端模型受限于固定上下文窗口，难以实现跨长时间交互的持久记忆。我们提出StateLinFormer，一种采用状态化记忆机制的线性注意力导航模型，在连续训练段间保留递归记忆状态，而非在每批次边界重置。这一训练范式有效逼近对无限长序列的学习，使模型具备长时程记忆保持能力。在MAZE与ProcTHOR环境中的实验表明，StateLinFormer显著优于其无状态的线性注意力基线和标准的固定上下文窗口Transformer模型。值得注意的是，随着交互长度增加，持续状态训练显著提升了上下文依赖的适应能力，表明其在导航任务中增强了上下文学习（ICL）能力。
原文摘要 · Abstract (English)
Effective navigation intelligence relies on long-term memory to support both immediate generalization and sustained adaptation. However, existing approaches face a dilemma: modular systems rely on explicit mapping but lack flexibility, while Transformer-based end-to-end models are constrained by fixed context windows, limiting persistent memory across extended interactions. We introduce StateLinFormer, a linear-attention navigation model trained with a stateful memory mechanism that preserves recurrent memory states across consecutive training segments instead of reinitializing them at each batch boundary. This training paradigm effectively approximates learning on infinitely long sequences, enabling the model to achieve long-horizon memory retention. Experiments across both MAZE and ProcTHOR environments demonstrate that StateLinFormer significantly outperforms its stateless linear-attention counterpart and standard Transformer baselines with fixed context windows. Notably, as interaction length increases, persistent stateful training substantially improves context-dependent adaptation, suggesting an enhancement in the model's In-Context Learning (ICL) capabilities for navigation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。