arXiv:2411.01159cs.LGcs.AI2024-11AAAI被引 1

将梯度提升与得分匹配结合,实现高效高精度的监督生成建模。

Supervised Score-Based Modeling by Gradient Boosting

  • 用得分匹配重构梯度提升,逐步优化数据分布估计。
  • 在多个数据集上准确率超越NGboost等模型,推理速度更快。
  • 适合需要概率输出与快速预测的机器学习任务。

得分生成模型通过估计数据分布的梯度来有效学习分布。由于其多步去噪特性,研究者近期尝试将得分生成模型与梯度提升算法(一种多步监督学习方法)结合,用于解决监督学习任务。然而,现有生成模型常受模型随机性及长推理时间影响,制约预测性能。为此,我们提出监督得分模型(SSM),可视为结合得分匹配的梯度提升算法。本文对SSM的学习与采样过程提供理论分析,以平衡推理时间和预测精度。通过选定案例的消融实验,验证了所提方法的卓越性能。此外,我们还将模型与自然梯度提升(NGboost)、分类与回归扩散模型(CARD)、扩散提升树(DBT)以及非概率梯度提升机(GBM)进行对比。实验结果表明,我们的模型在准确率和推理时间两方面均优于现有模型。

原文摘要 · Abstract (English)

Score-based generative models can effectively learn the distribution of data by estimating the gradient of the distribution. Due to the multi-step denoising characteristic, researchers have recently considered combining score-based generative models with the gradient boosting algorithm, a multi-step supervised learning algorithm, to solve supervised learning tasks. However, existing generative model algorithms are often limited by the stochastic nature of the models and the long inference time, impacting prediction performances. Therefore, we propose a Supervised Score-based Model (SSM), which can be viewed as a gradient boosting algorithm combining score matching. We provide a theoretical analysis of learning and sampling for SSM to balance inference time and prediction accuracy. Via the ablation experiment in selected examples, we demonstrate the outstanding performances of the proposed techniques. Additionally, we compare our model with other probabilistic models, including Natural Gradient Boosting (NGboost), Classification and Regression Diffusion Models (CARD), Diffusion Boosted Trees (DBT), and non-probabilistic GBM models. The experimental results show that our model outperforms existing models in both accuracy and inference time.

得分模型梯度提升生成建模概率预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。