用XGBoost构建表格数据生成模型,小数据用扩散模型,大数据用分层自回归模型。
XGenBoost: Synthesizing Small and Large Tabular Datasets with XGBoost
- 以XGBoost为评分估计器构建扩散模型,避免独热编码并支持混合类型数据。
- 在小数据集上生成质量优于现有神经与树基模型,训练成本更低。
- 适合需要高效生成真实世界混合类型表格数据的场景。
树集成模型如XGBoost因其归纳偏置少、超参数调优需求低和训练效率高,在混合类型表格数据的判别任务中广受青睐。我们提出,若正确利用这些特性,也可用于生成建模。为此,我们设计了基于XGBoost的两类生成模型:一是在小数据集上适用的去噪扩散隐式模型(DDIM),以XGBoost作为评分估计器;二是在大规模表格合成中适用的分层自回归模型,其条件分布由XGBoost分类器学习。扩散模型结合高斯与多项式扩散,天然处理类别分裂,无需独热编码,准确建模混合数据类型;自回归模型采用固定顺序分解,使用层次分类器引入数值特征的序数归纳偏置,并通过经验分位数函数实现去量化,以刻画现实数据的非连续性。在包含小规模与大规模数据集的两个基准测试中,所提架构在生成质量上超越先前神经与树基生成模型,且训练成本更低。
原文摘要 · Abstract (English)
Tree ensembles such as XGBoost are often preferred for discriminative tasks in mixed-type tabular data, due to their inductive biases, minimal hyperparameter tuning, and training efficiency. We argue that these qualities, when leveraged correctly, can make for better generative models as well. As such, we present XGenBoost, a pair of generative models based on XGBoost: i) a Denoising Diffusion Implicit Model (DDIM) with XGBoost as score-estimator suited for smaller datasets, and ii) a hierarchical autoregressive model whose conditionals are learned via XGBoost classifiers, suited for large-scale tabular synthesis. The architectures follow from the natural constraints imposed by tree-based learners, e.g., in the diffusion model, combining Gaussian and multinomial diffusion to leverage native categorical splits and avoid one-hot encoding while accurately modelling mixed data types. In the autoregressive model, we use a fixed-order factorization, a hierarchical classifier to impose ordinal inductive biases when modelling numerical features, and de-quantization based on empirical quantile functions to model the non-continuous nature of most real-world tabular datasets. Through two benchmarks, one containing smaller and the other larger datasets, we show that our proposed architectures outperform previous neural- and tree-based generative models for mixed-type tabular synthesis at lower training cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。