用小样本审计模型提升分类器在分布偏移下的置信度可靠性
Audited Conformal Prediction for Classification under Unknown Distribution Shift
- 引入审计模型识别旧模型易错输入，融合到置信区间生成中
- 实际条件覆盖率显著优于现有方法，理论保障边际与分组覆盖率
- 适合部署后需高可信预测的工业场景，尤其未知分布偏移时
针对预训练分类模型在未知分布偏移下进行不确定性量化的问题，本文提出审计性合规预测（ACP）。该方法利用目标域的小规模带标签数据训练一个辅助审计模型，用于识别旧模型可能出错的输入。通过将审计模型输出整合进合规预测框架，ACP 在保证边际覆盖率的前提下，实际中实现了显著更高的条件覆盖率。我们设计并分析了两种互补的集成策略：一种优化边际覆盖率并提升条件性能，另一种提供明确的组条件覆盖率保证，并为两者建立了理论保障。在合成与真实数据集上的实验验证了方法有效性，展示了预测集大小与条件覆盖率之间的权衡。
原文摘要 · Abstract (English)
We consider the problem of uncertainty quantification for a pretrained classification model deployed under unknown distribution shift. We propose Audited Conformal Prediction (ACP), a method that leverages a small labeled dataset from the target population to train an auxiliary audit model identifying inputs where the legacy model is likely to fail. By integrating the audit model's outputs into the conformal prediction framework, ACP produces prediction sets that guarantee marginal coverage while achieving substantially higher conditional coverage in practice than existing approaches. We develop and analyze two complementary integration strategies -- one targeting marginal coverage with improved conditional performance, the other providing explicit group-conditional coverage guarantees -- and establish theoretical guarantees for both. Experiments on synthetic and real-world datasets validate the method and illustrate trade-offs between prediction set size and conditional coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。