提出能量模型增强的树模型，既能分类又能生成数据。
NRGBoost: Energy-Based Generative Boosted Trees
- 用能量函数建模数据密度，实现生成与判别双任务。
- 在真实表格数据上分类性能媲美GBDT，采样能力超越传统生成方法。
- 适合需要同时做分类和生成的表格数据场景。
尽管深度学习在非结构化数据领域占据主导地位，随机森林（RF）和梯度提升决策树（GBDT）仍是处理表格数据判别任务的核心方法。本文探索这些流行算法的生成扩展，重点在于显式建模数据密度（至归一化常数），从而支持采样以外的应用。主要贡献是提出一种能量基础的生成提升算法，其形式类似于XGBoost等主流库中的二阶提升。实验表明，该算法虽能对任意输入变量进行推断，但在多个真实世界表格数据集上分类性能与GBDT相当，优于其他生成方法；同时在采样任务上也具备与神经网络模型竞争的能力。代码已开源：https://github.com/ajoo/nrgboost。
原文摘要 · Abstract (English)
Despite the rise to dominance of deep learning in unstructured data domains, tree-based methods such as Random Forests (RF) and Gradient Boosted Decision Trees (GBDT) are still the workhorses for handling discriminative tasks on tabular data. We explore generative extensions of these popular algorithms with a focus on explicitly modeling the data density (up to a normalization constant), thus enabling other applications besides sampling. As our main contribution we propose an energy-based generative boosting algorithm that is analogous to the second-order boosting implemented in popular libraries like XGBoost. We show that, despite producing a generative model capable of handling inference tasks over any input variable, our proposed algorithm can achieve similar discriminative performance to GBDT on a number of real world tabular datasets, outperforming alternative generative approaches. At the same time, we show that it is also competitive with neural-network-based models for sampling. Code is available at https://github.com/ajoo/nrgboost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。