通过可学习的特征变换，提升序列回归的性能与泛化能力
Soft Gradient Boosting with Learnable Feature Transforms for Sequential Regression
- 在每次迭代中联合训练软决策树与线性特征变换
- 高维小样本下显著提升性能，有效避免过拟合
- 支持可微非线性变换，适合数据稀缺场景
我们提出一种用于序列回归的软梯度提升框架，在提升过程中嵌入可学习的线性特征变换。每轮迭代中，同时训练软决策树并学习输入特征变换矩阵Q。该方法在高维、数据稀疏场景下表现优异，能自动发现最相关输入表示，通过端到端优化特征选择与提升过程，显著提升性能且避免过拟合。当过拟合不严重时，还可扩展至可微非线性变换。为保障可复现性，代码已公开。
原文摘要 · Abstract (English)
We propose a soft gradient boosting framework for sequential regression that embeds a learnable linear feature transform within the boosting procedure. At each boosting iteration, we train a soft decision tree and learn a linear input feature transform Q together. This approach is particularly advantageous in high-dimensional, data-scarce scenarios, as it discovers the most relevant input representations while boosting. We demonstrate, using both synthetic and real-world datasets, that our method effectively and efficiently increases the performance by an end-to-end optimization of feature selection/transform and boosting while avoiding overfitting. We also extend our algorithm to differentiable non-linear transforms if overfitting is not a problem. To support reproducibility and future work, we share our code publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。