MorphBoost让决策树在训练中动态变形，自适应调整分裂策略，提升模型性能与稳定性。
MorphBoost: Self-Organizing Universal Gradient Boosting with Adaptive Tree Morphing
- 树结构在训练中动态演化，分裂策略随梯度统计和学习进度自适应调整。
- 平均比XGBoost高0.84%，10个数据集上赢4次，方差最低（σ=0.0948）。
- 适合需要高鲁棒性和自适应能力的复杂分类与回归任务。
传统梯度提升算法使用固定结构的决策树，分裂规则在训练中不变，难以适应梯度分布变化和不同学习阶段的问题特征。本文提出MorphBoost框架，引入可自我组织的动态树结构，在训练过程中自适应地改变分裂行为。算法通过累积梯度统计与迭代依赖的学习压力，实现分裂函数的自适应演化。核心创新包括：(1) 结合梯度评分与信息论指标的动态分裂准则，权重随训练进程调整；(2) 自动问题指纹识别，智能配置二分类、多分类及回归任务参数；(3) 向量化预测，显著提升计算效率；(4) 交互感知特征重要性，检测特征间的乘积关系；(5) 快速模式优化，平衡速度与精度。在10个多样化数据集上对齐竞争模型（XGBoost、LightGBM、GradientBoosting、HistGradientBoosting、集成方法）的全面测试表明，MorphBoost达到当前最优性能，平均优于XGBoost 0.84%。其在10个数据集中赢得4个（胜率40%），30次排名中有6次进入前三（20%），且方差最低（σ=0.0948），最低准确率最高，展现出优异的一致性与鲁棒性。难度分析显示，在简单数据集上表现稳健，复杂任务中因更强自适应能力而显著提升。
原文摘要 · Abstract (English)
Traditional gradient boosting algorithms employ static tree structures with fixed splitting criteria that remain unchanged throughout training, limiting their ability to adapt to evolving gradient distributions and problem-specific characteristics across different learning stages. This work introduces MorphBoost, a new gradient boosting framework featuring self-organizing tree structures that dynamically morph their splitting behavior during training. The algorithm implements adaptive split functions that evolve based on accumulated gradient statistics and iteration-dependent learning pressures, enabling automatic adjustment to problem complexity. Key innovations include: (1) morphing split criterion combining gradient-based scores with information-theoretic metrics weighted by training progress; (2) automatic problem fingerprinting for intelligent parameter configuration across binary/multiclass/regression tasks; (3) vectorized tree prediction achieving significant computational speedups; (4) interaction-aware feature importance detecting multiplicative relationships; and (5) fast-mode optimization balancing speed and accuracy. Comprehensive benchmarking across 10 diverse datasets against competitive models (XGBoost, LightGBM, GradientBoosting, HistGradientBoosting, ensemble methods) demonstrates that MorphBoost achieves state-of-the-art performance, outperforming XGBoost by 0.84% on average. MorphBoost secured the overall winner position with 4/10 dataset wins (40% win rate) and 6/30 top-3 finishes (20%), while maintaining the lowest variance (σ=0.0948) and highest minimum accuracy across all models, revealing superior consistency and robustness. Performance analysis across difficulty levels shows competitive results on easy datasets while achieving notable improvements on advanced problems due to higher adaptation levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。