让分类器和拒答策略一起训练，提升准确率与可信度。
Joint Training for Selective Prediction
- 联合优化分类器与拒答策略的表示学习
- 在4个任务上优于两个强基线，双方性能均提升
- 适合需要人机协作的高可靠场景
分类模型在自然语言处理中广泛应用，准确率较高。但在实际应用中，人机协同系统可通过信任模型输出实现更高性能。选择性预测（Selective Prediction, SP）方法决定何时采纳模型输出，何时交由人工判断。以往方法或改进softmax作为置信度度量，或构建独立的置信度估计器。一种方法基于人工设计特征学习拒答模型。本文提出一种新型联合训练方法，同时优化分类模块所用的可学习表示与可学习的拒答策略。在四个分类任务上的实验表明，该方法不仅显著优于两个强基线的SP表现，还提升了两个模块自身的性能。
原文摘要 · Abstract (English)
Classifier models are prevalent in natural language processing (NLP), often with high accuracy. Yet in real world settings, human-in-the-loop systems can foster trust in model outputs and even higher performance. Selective Prediction (SP) methods determine when to adopt a classifier's output versus defer to a human. Previous SP approaches have addressed how to improve softmax as a measure of model confidence, or have developed separate confidence estimators. One previous method involves learning a deferral model based on engineered features. We introduce a novel joint-training approach that simultaneously optimizes learned representations used by the classifier module and a learned deferral policy. Our results on four classification tasks demonstrate that joint training not only leads to better SP outcomes over two strong baselines, but also improves the performance of both modules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。