提出新指标'跨样本预测波动'，并给出有效降低方法。
Reducing cross-sample prediction churn in scientific machine learning
- 用双bootstrap联合训练+对称KL损失约束预测一致性
- 在9个化学基准上使预测分歧率降低40%~54%
- 适合关注模型稳定性的科学机器学习研究者
科学机器学习报告预测性能，但未说明相同预测在不同训练数据抽样下是否一致。在9个化学基准上，两个基于独立自助采样的分类器在整体准确率上相差仅1.3–4.2个百分点，但在测试分子的类别标签上存在8.0–21.8%的分歧，我们称之为‘跨样本预测波动’。标准参数侧方法（如深度集成、MC dropout、随机权重平均）无法缓解此问题；而两种数据侧方法有效。一是K-Bootstrap袋装法，在不牺牲准确率的情况下将波动率降低40%–54%（代价为K×-ERM计算）。二是本文提出的‘孪生自助法’：两网络在独立自助样本上联合训练，通过预测间的对称KL一致性损失进行约束，在匹配2×-ERM计算开销下，相较bagging-K=2进一步降低中位45%的波动。跨样本预测波动应作为科学机器学习基准报告中的常规指标，否则参数与数据侧方法在实际差异点上无法区分。
原文摘要 · Abstract (English)
Scientific machine learning reports predictive performance. It does not report whether the same prediction would survive a different draw of training data. Across $9$ chemistry benchmarks, two classifiers trained on independent bootstraps of the same training set agree on aggregate accuracy to within $1.3\text{--}4.2$ percentage points but disagree on the class label of $8.0\text{--}21.8\%$ of test molecules. We call this gap \emph{cross-sample prediction churn}. The standard parameter-side techniques (deep ensembles, MC dropout, stochastic weight averaging) do not reduce this gap; two data-side methods do. The first is $K$-bootstrap bagging, which cuts the rate $40\text{--}54\%$ on every dataset at no accuracy cost ($K{\times}$-ERM compute). The second is \emph{twin-bootstrap}, our proposal: two networks trained jointly on independent bootstraps with a sym-KL consistency loss between their predictions, which at matched $2{\times}$-ERM compute reduces churn a further median $45\%$ beyond bagging-$K{=}2$. Cross-sample prediction churn deserves a column alongside predictive performance in scientific-ML benchmark reports, because without it the parameter-side and data-side methods are indistinguishable on the metric they actually differ on.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。