通过历史增益筛选高价值特征，加速梯度提升树训练
EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening
- 用指数移动平均记录特征分裂增益，动态筛选高贡献特征
- 在保留30%特征时实现2.61倍加速，70%保留下速度提升1.34倍
- 兼容原版LightGBM，适合高维密集数据场景的高效训练
梯度提升决策树（如LightGBM）训练中约65-70%时间用于构建每特征直方图。现有方法如随机特征子采样不考虑特征预测能力。本文提出基于指数移动平均的特征筛选（EMA-FS），在多个迭代后仅对历史增益最高的前K个特征构建直方图。相比随机采样，该方法能保留高增益特征、剔除低效特征。该策略在树级别运行，兼容LightGBM的直方图相减技巧，无需修改核心代码。在金融欺诈检测、广告点击率预测等数据集上测试，特征维度29至968。在密集高维数据上取得显著加速：500维合成数据上达2.61倍，IEEE-CIS欺诈数据（432维）在30%保留率下达1.45倍；70%保留率时AUC提升0.11点，速度提升1.34倍。在极稀疏数据（Bosch，缺失率>90%）上无加速，因LightGBM已优化空值处理。进一步提出随机版EMA-FS（S-EMA-FS），以浓度参数beta统一确定性筛选与随机采样，二者均以约120行C++代码集成至LightGBM所有六类树学习器，完全向后兼容。
原文摘要 · Abstract (English)
Gradient Boosted Decision Trees (GBDT), exemplified by LightGBM, spend a dominant fraction of training time -- typically 65-70% -- constructing per-feature histograms. Existing approaches such as random feature subsampling (feature_fraction) discard features without regard for their predictive utility. We propose EMA-based Feature Screening (EMA-FS), an algorithm-level optimization that maintains an exponential moving average (EMA) of per-feature split gains across boosting iterations and, after a short warmup, restricts histogram construction to the top-K features ranked by historical gain. Unlike random subsampling, EMA-FS is informed: it retains high-gain features while screening out low-gain ones. Operating at the per-tree level, it preserves full compatibility with LightGBM's histogram subtraction trick, requiring no changes to core routines. We evaluate EMA-FS on datasets spanning financial fraud detection, advertising click-through prediction, industrial quality control, and synthetic benchmarks, with feature dimensionalities from 29 to 968. On dense, moderate-to-high-dimensional data it achieves significant speedups: 2.61x on a 500-feature synthetic benchmark and 1.45x on the 432-feature IEEE-CIS Fraud dataset at 30% retention. At 70% retention it improves AUC by 0.11 points while delivering a 1.34x speedup. On extremely sparse data (Bosch, >90% missing) it yields no speedup, as LightGBM's sparse bin optimization already bypasses empty values. We further introduce Stochastic EMA-FS (S-EMA-FS), which replaces deterministic top-K selection with gain-weighted random sampling controlled by a concentration parameter beta, unifying deterministic EMA-FS (beta -> infinity) and random subsampling (beta = 0) in one framework. Both are implemented in ~120 lines of C++ across all six LightGBM tree learners and are fully backward-compatible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。