条件推理森林可高效筛选关键特征,适合高维数据建模。
Conditional Inference Trees and Forests for Feature Selection

- 基于置换检验选择分裂点,降低特征选择偏差。
- 在22个分类和8个回归数据集上表现优于多数方法。
- 自适应停止与阈值搜索对计算效率影响显著。
条件推理树(CIT)和条件推理森林(CIF)通过在选择分裂阈值前测试特征,减少分裂选择偏差,但重复的置换检验和阈值搜索使其计算成本较高。本文将CIT和CIF作为下游预测的top-k特征排序方法,在真实数据基准、运行时消融实验和合成特征恢复实验中进行评估。在固定节点下,若特征与置换预算不依赖节点响应,贝叶斯校正+1蒙特卡洛置换p值可控制节点层面拒绝率。在22个分类数据集上CIF排名第四(共17种方法),在8个回归数据集上排名第三(共18种方法)。固定贝叶斯校正时,运行时消融显示:关闭自适应停止和使用精确阈值搜索分别使拟合时间增加4.0–8.4倍和1.9–10.8倍,而下游得分变化不超过0.011。稀疏高维模拟表明,森林特征采样可能导致重要特征被忽略。结果支持CIF作为评估范围内有效的top-k特征排序方法。
原文摘要 · Abstract (English)
Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive. We study CIT and CIF as top-$k$ feature-ranking methods for downstream prediction using real-data benchmarks, runtime ablations, and synthetic feature-recovery experiments. At a fixed node, if the features and permutation budget do not depend on the node responses, Bonferroni-corrected $+1$ Monte Carlo permutation $p$-values control nodewise rejection under the complete permutation null. CIF ranks 4th among 17 classification methods on 22 datasets and 3rd among 18 regression methods on 8 datasets. With Bonferroni correction held fixed, the CIF runtime ablations indicate that adaptive stopping and the number of thresholds searched have the largest measured effect on runtime: turning off adaptive stopping and using exact threshold search increase fitting time by 4.0--8.4$\times$ and 1.9--10.8$\times$, respectively, while downstream score changes are at most 0.011. Sparse high-$p$ simulations indicate that forest feature sampling can leave informative features out of many split decisions. Overall, the results support CIF as a top-$k$ feature-ranking method in the evaluated downstream prediction benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。