通过改进预测模型部署失败率的方法，提升安全评估可靠性。
Training ML Models with Predictable Failures
- 基于评估集最大k个错误得分外推部署失败率
- 发现原有方法在多数情况下会高估失败率（偏保守）
- 提出可微分的预报性损失函数，降低预测偏差
估计机器学习模型在实际部署中的失败频率是预部署安全评估的核心问题，但可用评估数据集通常太小，难以观测关键失败。Jones等（2025）通过外推评估集中最大的k个失败得分来预测部署规模下的失败率。本文对这一估计器的预测误差进行有限k分解，表明其在典型情况下存在内置偏差，倾向于高估失败率（有利于安全）。当评估集遗漏了部署集中存在的罕见高失败模式时，该偏差消失，导致预测值低于真实值。为此，我们提出一种细调目标——预报性损失（forecastability loss），有效缓解此问题。在两个概念验证实验中（语言模型密码游戏和强化学习网格世界），使用该损失进行微调显著降低了保留集上的预测误差，同时保持主任务性能，并达到与监督基线相当的安全水平。
原文摘要 · Abstract (English)
Estimating how often an ML model will fail at deployment scale is central to pre-deployment safety assessment, but a feasible evaluation set is rarely large enough to observe the failures that matter. Jones et al. (2025) address this by extrapolating from the largest k failure scores in an evaluation set to predict deployment-scale failure rates. We give a finite-k decomposition of this estimator's forecast error and show that it has a built-in bias toward over-prediction in the typical case, which is the safety-favorable direction. This bias is offset when the evaluation set misses a rare high-failure mode that the deployment set contains, leaving the forecast to under-predict at deployment scale. We propose a fine-tuning objective, the forecastability loss, that addresses this failure mode. In two proof-of-concept experiments, a language-model password game and an RL gridworld, fine-tuning substantially reduces held-out forecast error while preserving primary-task capability and achieving safety similar to that of supervised baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。