提出可微且透明的流数据树模型,兼顾学习能力与解释性。
Soft Hoeffding Tree: A Transparent and Differentiable Model on Data Streams
- 引入新门控函数平衡单变量与多变量分裂,实现可微分结构
- 在20个数据流上优于传统霍夫丁树,准确率更高且保持透明性
- 支持通过超参数灵活权衡透明性与性能,适合需要解释性的实时系统
我们提出软霍夫丁树(SoHoT)作为一种新的可微分且透明的模型,用于处理可能无限且动态变化的数据流。现有的流挖掘算法如霍夫丁树基于输入数据流进行生长,但缺乏端到端深度学习系统的适应性。若特征表示由神经网络学习并用于树结构,或树输出进一步接入深度学习模型,则端到端学习具有优势。不同于霍夫丁树,软树虽可集成于此类系统,但缺乏透明性与可解释性。本文模型结合了霍夫丁树的可扩展性与透明性,以及软树的可微性。引入新门控函数调控单变量与多变量分裂间的平衡。在20个数据流上对比标准霍夫丁树、复杂度受限的霍夫丁树及使用稀疏激活函数进行样本路由的软树,结果表明:软霍夫丁树在估计类别概率方面优于传统霍夫丁树,同时相较于软树保持更高的透明性,仅在AUROC和交叉熵上有小幅损失。还展示了通过超参数控制透明性与性能的权衡,一端为单变量分裂,另一端为多变量分裂。
原文摘要 · Abstract (English)
We propose soft Hoeffding trees (SoHoT) as a new differentiable and transparent model for possibly infinite and changing data streams. Stream mining algorithms such as Hoeffding trees grow based on the incoming data stream, but they currently lack the adaptability of end-to-end deep learning systems. End-to-end learning can be desirable if a feature representation is learned by a neural network and used in a tree, or if the outputs of trees are further processed in a deep learning model or workflow. Different from Hoeffding trees, soft trees can be integrated into such systems due to their differentiability, but are neither transparent nor explainable. Our novel model combines the extensibility and transparency of Hoeffding trees with the differentiability of soft trees. We introduce a new gating function to regulate the balance between univariate and multivariate splits in the tree. Experiments are performed on 20 data streams, comparing SoHoT to standard Hoeffding trees, Hoeffding trees with limited complexity, and soft trees applying a sparse activation function for sample routing. The results show that soft Hoeffding trees outperform Hoeffding trees in estimating class probabilities and, at the same time, maintain transparency compared to soft trees, with relatively small losses in terms of AUROC and cross-entropy. We also demonstrate how to trade off transparency against performance using a hyperparameter, obtaining univariate splits at one end of the spectrum and multivariate splits at the other.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。