无需访问模型内部，即可优化黑盒分类器的预测结果。
An Efficient Plugin Method for Metric Optimization of Black-Box Models
- 通过后处理方式调整黑盒模型输出，适配目标数据分布。
- 仅需少量概率预测和真实标签，就能优化混淆矩阵中的特定指标。
- 适用于表格与语言任务，对模型无修改且效果媲美先进方法。
许多机器学习算法和分类器只能通过API调用作为“黑盒”使用——下游用户无法更改、重训或微调模型以适应特定目标分布。事实上，用户甚至可能不了解原始训练分布或用于构建和优化黑盒模型的性能指标。我们提出一种简单高效的后处理方法Plugin，可对任意黑盒分类器的多分类预测进行调整，同时实现（1）使预测适配目标分布；（2）优化混淆矩阵中的特定指标。重要的是，Plugin是一种完全后验的方法，不依赖特征信息，仅需少量概率预测及其对应的真实标签，并通过查询实现指标优化。我们在多种表格和语言任务上实证表明，Plugin具有广泛适用性，且性能与现有方法相当。
原文摘要 · Abstract (English)
Many machine learning algorithms and classifiers are available only via API queries as a ``black-box'' -- that is, the downstream user has no ability to change, re-train, or fine-tune the model on a particular target distribution. Indeed, the downstream user may not even have knowledge of the \emph{original} training distribution or performance metric used to construct and optimize the black-box model. We propose a simple and efficient method, Plugin, which \emph{post-processes} arbitrary multiclass predictions from any black-box classifier in order to simultaneously (1) adapt these predictions to a target distribution; and (2) optimize a particular metric of the confusion matrix. Importantly, Plugin is a completely \textit{post-hoc} method which does not rely on feature information, only requires a small amount of probabilistic predictions along with their corresponding true label, and optimizes metrics by querying. We empirically demonstrate that Plugin is both broadly applicable and has performance competitive with related methods on a variety of tabular and language tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。