Markov边界能显著提升表格预测,但现有方法难以有效利用。
The Good, the Bad, and the Ugly of Markov Boundary for Tabular Prediction
- 用因果发现提取真实边界,可大幅改善预测性能
- 特征空间越大越稀疏,边界优势越明显,最高提升达23.4%
- 现有方法因结构误判和计算不足,难超越全特征集
在标准图模型假设下,目标变量的Markov边界是使其他所有特征冗余的最小特征集。一旦观察到该边界,目标与表中其余列即条件独立。这为表格预测提供了理想候选特征集。然而,现代回归器仍基于全特征训练。本文在包含3,450个任务的SCM3K合成基准上评估了这一假设,其特征数在40至1,000之间,涵盖六类生成模型,使用六种回归器进行测试。结果表明,将回归器限制在理想边界(oracle boundary)通常能显著提升预测性能,且在高维稀疏场景下提升更明显。但通过因果发现恢复边界再训练的自然流程并未成功:现有估计器在达到最有利区间前已耗尽计算资源,即使运行也极少优于全特征集。原因有三:因果发现优化结构恢复而非预测;误删与误增特征带来的预测代价不对称;真实边界只是众多优于全特征集的子集之一。由此揭示出面向预测的特征选择新范式及学习因果结构的表格模型设计方向。
原文摘要 · Abstract (English)
Under standard graphical assumptions, the Markov boundary of a target variable is the smallest set of features that renders every other feature redundant. Once the boundary is observed, the target is conditionally independent of the rest of the table. This is a tempting object for tabular prediction, since it names exactly the columns a model should need. Yet modern regressors are still trained on the full feature set. We ask whether the Markov boundary is genuinely useful for prediction on SCM3K, a 3,450-task synthetic SCM benchmark with feature counts from 40 to 1000 and six SCM families, evaluated with six regressors. The answer is more nuanced than the theory suggests. Restricting a regressor to the oracle boundary often improves prediction substantially, and the improvement grows as the feature space becomes larger and sparser. But the natural pipeline of recovering the boundary with causal discovery and training on the recovered mask does not deliver. Existing estimators exhaust the compute budget before reaching the regime where the boundary helps most, and even where they run they rarely beat the full feature set. We trace this to three causes. Discovery optimizes structural recovery rather than prediction. False negatives and false positives carry sharply asymmetric predictive cost. The exact boundary is only one of many feature sets that beat all features. We then develop what these facts imply for prediction-aligned feature selection and for tabular models that learn to use causal structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。