用树模型驱动的隐空间扩散法,提升表格数据少数类预测性能。
Boosting Predictive Performance on Tabular Data through Data Augmentation with Latent-Space Flow-Based Diffusion
- 在隐空间用梯度提升树学习向量场,保持表格结构并降低计算开销。
- AttentionForest在11个数据集上平均少数类召回率最优,且生成速度快于基线。
- 方法兼顾隐私保护与生成质量,适合医疗、金融等敏感领域应用。
真实世界表格学习中严重类别不平衡普遍存在,少数但关键类别对可靠预测至关重要。现有生成过采样方法如GAN、VAE和扩散模型虽能提升少数类表现,但常面临表格异质性、训练不稳定和隐私问题。本文提出一类基于隐空间、树驱动的扩散方法,采用条件流匹配,以梯度提升树作为向量场学习器。模型在紧凑隐空间中运行,保留表格结构并减少计算量。提出三种变体:PCAForest使用线性主成分嵌入;EmbedForest使用学习的非线性嵌入;AttentionForest使用注意力增强嵌入。每种方法将GBT-based流与解码器结合,映射回原始特征空间。在医疗、金融、制造领域的11个数据集上,AttentionForest实现最高平均少数类召回率,同时保持良好的精确率、校准性和分布相似性。PCAForest与EmbedForest在生成速度上显著更快,提供更优的准确率-效率权衡。隐私评估(最近邻距离比与到最近记录距离)优于或相当基线ForestDiffusion。消融实验表明,较小嵌入有助于提升少数类召回,而激进的学习率会损害稳定性。总体而言,该方法在严重类别不平衡下实现了高效、隐私友好的高保真表格数据增强。
原文摘要 · Abstract (English)
Severe class imbalance is common in real-world tabular learning, where rare but important minority classes are essential for reliable prediction. Existing generative oversampling methods such as GANs, VAEs, and diffusion models can improve minority-class performance, but they often struggle with tabular heterogeneity, training stability, and privacy concerns. We propose a family of latent-space, tree-driven diffusion methods for minority oversampling that use conditional flow matching with gradient-boosted trees as the vector-field learner. The models operate in compact latent spaces to preserve tabular structure and reduce computation. We introduce three variants: PCAForest, which uses linear PCA embedding; EmbedForest, which uses a learned nonlinear embedding; and AttentionForest, which uses an attention-augmented embedding. Each method couples a GBT-based flow with a decoder back to the original feature space. Across 11 datasets from healthcare, finance, and manufacturing, AttentionForest achieves the best average minority recall while maintaining competitive precision, calibration, and distributional similarity. PCAForest and EmbedForest reach similar utility with much faster generation, offering favorable accuracy-efficiency trade-offs. Privacy evaluated with nearest-neighbor distance ratio and distance-to-closest-record is comparable to or better than the ForestDiffusion baseline. Ablation studies show that smaller embeddings tend to improve minority recall, while aggressive learning rates harm stability. Overall, latent-space, tree-driven diffusion provides an efficient and privacy-aware approach to high-fidelity tabular data augmentation under severe class imbalance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。