动态LSTM框架解决流数据特征维度变化问题，支持持续学习不遗忘。
packetLSTM: Dynamic LSTM Framework for Streaming Data with Varying Feature Space
- 为每个输入特征配置独立LSTM，共享内存整合全局信息
- 在5个数据集上达到当前最优性能，支持在线增删特征
- 适用于实时流数据，特别适合特征维度频繁变化的场景
我们研究了流数据中输入特征空间动态变化的在线学习问题。尽管LSTM能有效捕捉流数据的时间特性，但在在线学习场景下无法处理维度变化的流数据。为此，我们提出一种基于动态LSTM的新方法——packetLSTM，用于建模维度变化的流数据。其动态框架由一组不断演化的LSTM组成，每个LSTM专门处理一个输入特征。每个LSTM保留对应特征的局部信息，而共享公共记忆则整合全局信息。该结构支持持续学习，即使某些特征长时间缺失也能避免遗忘。通过为每个特征配置一个LSTM，并使用维度不变的聚合算子，提升了packetLSTM的动态性。这种动态性体现在模型可按需激活、停用或添加新的LSTM，从而无缝适应输入维度的变化。packetLSTM在五个数据集上取得了最先进的结果，其核心思想还可扩展至GRU和普通RNN等其他RNN类型。
原文摘要 · Abstract (English)
We study the online learning problem characterized by the varying input feature space of streaming data. Although LSTMs have been employed to effectively capture the temporal nature of streaming data, they cannot handle the dimension-varying streams in an online learning setting. Therefore, we propose a dynamic LSTM-based novel method, called packetLSTM, to model the dimension-varying streams. The packetLSTM's dynamic framework consists of an evolving packet of LSTMs, each dedicated to processing one input feature. Each LSTM retains the local information of its corresponding feature, while a shared common memory consolidates global information. This configuration facilitates continuous learning and mitigates the issue of forgetting, even when certain features are absent for extended time periods. The idea of utilizing one LSTM per feature coupled with a dimension-invariant operator for information aggregation enhances the dynamic nature of packetLSTM. This dynamic nature is evidenced by the model's ability to activate, deactivate, and add new LSTMs as required, thus seamlessly accommodating varying input dimensions. The packetLSTM achieves state-of-the-art results on five datasets, and its underlying principle is extended to other RNN types, like GRU and vanilla RNN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。