用随机组合列条件构建深度决策树，提升表格数据上的表现
DOFEN: Deep Oblivious Forest ENsemble
- 用随机组合列条件构造宽松的遗忘决策树
- 在73个表格数据集上达到当前最优的DNN性能
- 适合处理表格数据且想超越传统树模型的研究者
深度神经网络（DNN）在图像、视频和文本等数据类型上取得了显著成果，但在表格数据上的表现仍落后于梯度提升决策树（GBDT）。本文提出DOFEN（Deep Oblivious Forest ENsemble），一种受遗忘决策树启发的新颖DNN架构。DOFEN通过随机组合每列的条件构建松弛遗忘决策树（rODTs），并采用两级rODT森林集成机制进一步提升性能。该方法在公认的基准测试Tabular Benchmark（涵盖73个来自多个领域的数据集）上实现了当前DNN的最佳表现，显著缩小了DNN与基于树模型之间的差距。代码已开源：https://github.com/Sinopac-Digital-Technology-Division/DOFEN。
原文摘要 · Abstract (English)
Deep Neural Networks (DNNs) have revolutionized artificial intelligence, achieving impressive results on diverse data types, including images, videos, and texts. However, DNNs still lag behind Gradient Boosting Decision Trees (GBDT) on tabular data, a format extensively utilized across various domains. In this paper, we propose DOFEN, short for \textbf{D}eep \textbf{O}blivious \textbf{F}orest \textbf{EN}semble, a novel DNN architecture inspired by oblivious decision trees. DOFEN constructs relaxed oblivious decision trees (rODTs) by randomly combining conditions for each column and further enhances performance with a two-level rODT forest ensembling process. By employing this approach, DOFEN achieves state-of-the-art results among DNNs and further narrows the gap between DNNs and tree-based models on the well-recognized benchmark: Tabular Benchmark \citep{grinsztajn2022tree}, which includes 73 total datasets spanning a wide array of domains. The code of DOFEN is available at: \url{https://github.com/Sinopac-Digital-Technology-Division/DOFEN}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。