用置信集挑选最优专家组合，提升人机协作分类效果
Conformal Set-based Human-AI Complementarity with Multiple Experts
- 基于置信集动态筛选每例任务的最优专家子集
- 在CIFAR-10H和ImageNet-16H上接近最优性能
- 适合多专家协作的智能决策系统设计
决策支持系统通过预训练模型生成的置信集辅助人类专家进行分类。本文聚焦于从多个专家中为每个实例选择最合适的专家组合，而非传统单专家场景。我们分析了多专家在置信集下获益的条件，提出一种利用置信集的贪心算法，以识别对当前实例分类最有价值的专家预测子集。基于真实专家在CIFAR-10H与ImageNet-16H上的标注数据，模拟实验表明该方法可获得近似最优的专家子集，显著提升多专家协作下的分类表现。
原文摘要 · Abstract (English)
Decision support systems are designed to assist human experts in classification tasks by providing conformal prediction sets derived from a pre-trained model. This human-AI collaboration has demonstrated enhanced classification performance compared to using either the model or the expert independently. In this study, we focus on the selection of instance-specific experts from a pool of multiple human experts, contrasting it with existing research that typically focuses on single-expert scenarios. We characterize the conditions under which multiple experts can benefit from the conformal sets. With the insight that only certain experts may be relevant for each instance, we explore the problem of subset selection and introduce a greedy algorithm that utilizes conformal sets to identify the subset of expert predictions that will be used in classifying an instance. This approach is shown to yield better performance compared to naive methods for human subset selection. Based on real expert predictions from the CIFAR-10H and ImageNet-16H datasets, our simulation study indicates that our proposed greedy algorithm achieves near-optimal subsets, resulting in improved classification performance among multiple experts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。