将梯度提升扩展到向量输出，支持多分类等复杂任务
Gradient boosting with vector-valued leafs
- 用向量值叶子替代标量叶子，直接优化多维目标函数
- 提出高效算法，适配基于直方图的决策树结构
- 适合需要多标签或多类别输出的机器学习场景
梯度提升通过决策树集成在多种问题上取得成功，通常使用基于单变量似然的简单目标函数。该概念自然可推广至向量输入的目标函数，例如多分类中的多项式逻辑似然，此时每个样本对各分类有独立得分。然而，主流框架通常逐个更新输入向量分量，或采用二阶导数的对角上界近似。本文将传统梯度提升框架扩展至向量输入函数，并提出一种可高效配合基于直方图的决策树的简单算法。
原文摘要 · Abstract (English)
Gradient boosting in the form of decision tree ensembles has successfully been applied to a variety of problems using simple objective functions based on log-likelihoods of a single variable. The concept extends naturally to objective functions operating on vectors - for example, multinomial logistic log-likelihood for multi-class classification, where observations have a score for each class - but popular frameworks approach these functions by either updating one value of the input vectors at a time, or by using a diagonal upper bound on the second derivative. This work extends the usual gradient boosting framework to functions of vector inputs and sketches a simple algorithm that can be used efficiently with histogram-based decision trees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。