为表格数据的难例打分,提升模型预测可靠性。
Trajectory-Based Difficulty Scoring for Reliable Learning on Tabular Data

- 基于树模型的预测轨迹计算实例难度得分。
- 在多种数据集上与误差高度相关,优于现有方法。
- 适合需要精准识别难例的场景,如主动学习和风险控制。
梯度提升树在表格数据上表现优异,但常存在大量预测不佳的实例。本文提出轨迹难度评分(TDS),一种基于每棵树累积预测轨迹的实例级难度估计方法。对每个实例,计算可解释的轨迹特征(如方差、振荡峰值、符号变化、尾部稳定性),并训练轻量回归模型预测保留损失。通过经验累积分布函数校准,输出[0,1]区间内的难度分数,支持难例排序。在多个表格基准和集成规模下,TDS与误差具有强排名相关性,在分类任务中优于现有实例难易度和不确定性基线,回归任务也保持竞争力。进一步展示单一难度信号如何改进多种数据挖掘流程:基于难度的主动学习实现标签高效训练,难度阈值选择性预测优化风险-覆盖率权衡,以及基于TDS分层(Mondrian)的置信区间预测实现更均匀的条件覆盖。最后,利用SHAP归因对高TDS实例聚类,揭示由紧凑特征取值范围定义的连贯失败模式,支持错误分析与定向数据获取。
原文摘要 · Abstract (English)
Gradient-boosted trees achieve strong performance on tabular data, yet often leave a long tail of poorly predicted instances. We introduce a Trajectory-based Difficulty Score (TDS), an instance-level difficulty estimator for boosted ensembles derived from per-tree cumulative prediction trajectories. For each instance, we compute interpretable trajectory descriptors (e.g., variance, oscillation peaks, sign switches, and tail stability) and train a lightweight regression model to predict held-out loss. An empirical CDF calibrates the resulting signal into a score in $[0,1]$ that supports ranking hard cases. Across diverse tabular benchmarks and ensemble sizes, TDS exhibits strong rank correlation with error and outperforms established instance-hardness and uncertainty baselines on classification, while remaining competitive on regression. We then show how a single difficulty signal improves multiple data mining workflows: difficulty-driven active learning for label-efficient training, difficulty-thresholded selective prediction for improved risk-coverage trade-offs, and TDS-stratified (Mondrian) conformal prediction for more uniform conditional coverage. Finally, clustering high-TDS instances using SHAP attributions reveals coherent failure modes characterized by compact feature-value ranges, supporting error analysis and targeted data acquisition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。