XGBoost的特征子采样会阻碍模型学习比率类关系，影响性能。
Intra-tree Column Subsampling Hinders XGBoost Learning of Ratio-like Interactions
- 通过合成多个特征的分裂来隐式学习比率关系
- 当子采样率低至0.4时，测试PR-AUC下降54%
- 若已知比率结构，应避免子采样或直接加入比率特征
许多实际问题中，信号需通过组合多个原始测量值才能显现，如比率和速率。在梯度提升树中，这种组合并非显式操作，需通过各成分特征的协同分裂来实现。本文研究XGBoost中树内特征子采样对这种合成能力的影响。采用两种具有抵消结构的合成数据生成过程：两个原始特征共享强干扰因子，目标依赖于微小差异因子，而对数比率可消除干扰并凸显信号。实验中调整colsample_bylevel与colsample_bynode为{0.4, 0.6, 0.8, 0.9}，重点关注较弱子采样（s ≥ 0.8）。对比组包含人工构造的比率特征，无需合成。在两种过程中，仅使用原始特征时，树内子采样均降低测试集PR-AUC；主过程中两者均为0.4时相对下降达54%。当引入人工比率特征后，该影响基本消失。路径级共使用度量在性能下降区域同步下降。实践中，若比率结构合理，应避免树内子采样或直接加入目标比率特征。
原文摘要 · Abstract (English)
Many applied problems contain signal that becomes clear only after combining multiple raw measurements. Ratios and rates are common examples. In gradient boosted trees, this combination is not an explicit operation: the model must synthesize it through coordinated splits on the component features. We study whether intra-tree column subsampling in XGBoost makes that synthesis harder. We use two synthetic data generating processes with cancellation-style structure. In both, two primitive features share a strong nuisance factor, while the target depends on a smaller differential factor. A log ratio cancels the nuisance and isolates the signal. We vary colsample_bylevel and colsample_bynode over s in {0.4, 0.6, 0.8, 0.9}, emphasizing mild subsampling (s >= 0.8). A control feature set includes the engineered ratio, removing the need for synthesis. Across both processes, intra-tree column subsampling reduces test PR-AUC in the primitives-only setting. In the main process the relative decrease reaches 54 percent when both parameters are set to 0.4. The effect largely disappears when the engineered ratio is present. A path-based co-usage metric drops in the same cells where performance deteriorates. Practically, if ratio-like structure is plausible, either avoid intra-tree subsampling or include the intended ratio features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。