让决策树学会记忆，用梯度法训练可捕捉长期时序依赖的递归模型。
Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory
- 引入带内部记忆的递归决策树架构，类比RNN处理序列数据。
- 通过梯度下降优化轴对齐的决策规则，实现输出与状态同步更新。
- 适合需要可解释性又需建模时序模式的任务，如金融、医疗预测。
神经网络如循环神经网络（RNN）、Transformer 和状态空间模型在处理序列数据方面表现出色，能够学习时间依赖关系。相比之下，决策树（DT）虽广泛用于结构化表格数据，但通常无法直接捕捉序列模式。现有基于决策树的时间序列方法多依赖人工特征工程，如手动添加滞后特征，难以有效建模复杂时序依赖。为解决此问题，我们提出 ReMeDe Trees——一种新型递归决策树架构，集成类似 RNN 的内部记忆机制，以学习序列数据中的长期依赖。该模型通过梯度下降高效优化硬性、轴对齐的决策规则，同时用于输出生成和状态更新。我们在合成基准上进行了概念验证研究，证明了该方法的有效性。
原文摘要 · Abstract (English)
Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。