提出新方法精准控制模型决策的覆盖率与操作表现之间的权衡。
Conformal Tradeoffs: Operational Profiles Beyond Coverage
- 基于小样本贝塔校正,实现部署规则的有限样本覆盖率精确推断。
- 通过独立审计划分估算关键指标,支持可复用的运营性能分析。
- 适用于关注决策频率、拒绝率和错误暴露的工业级部署场景。
分位化预测在交换性假设下提供精确的有限样本覆盖保证,但实际部署中还需关注承诺频率、拒答率和决定性错误暴露等操作指标。这些指标不完全由覆盖率决定:相同覆盖率的校准方案可能带来截然不同的运营表现。本文在二分类分治分位化预测且固定部署规则的设定下,提出小样本贝塔校正(SSBC),通过反演贝塔-贝塔二项分布规律,将用户请求的(α⋆, δ)映射为最不保守的校准网格点,确保部署规则具备校准条件下的概率保证。校准-审计框架先通过校准确定规则,再使用独立审计子集估计区域-类别表,该表可投影得到各类运营关键绩效指标(KPI)。在此设计下,固定运营率支持精确的有限样本二项分布推断,而贝塔-二项分布包络线可用于未来窗口的实用预测。诱导出的划分还揭示了制度边界、帕累托相关权衡及固定下游约定下的逆定价问题。模拟验证了SSBC语义,并对比了审计摘要与留一法规划代理;分子毒性数据提供了审计实证案例,溶解度案例研究展示了在覆盖率确定后的场景规划能力。
原文摘要 · Abstract (English)
Conformal prediction gives exact finite-sample coverage guarantees under exchangeability, but deployed systems are judged by more than coverage alone. For a fixed calibrated rule reused over a finite operational window, stakeholders also care about deployment-facing quantities such as commitment frequency, deferral, and decisive error exposure. These are not determined by coverage: calibration choices with similar coverage can still induce materially different operational profiles. We study this characterization gap in a scoped setting: binary split conformal prediction under exchangeability with a fixed deployed rule. We introduce the Small-Sample Beta Correction (SSBC) which gives finite-sample coverage semantics for the deployed rule: it inverts the Beta/Beta--Binomial law governing calibration-conditional coverage to map a user request $(α^\star,δ)$ to the least conservative calibration grid point with calibration-conditional PAC semantics for the realized deployed rule. Calibrate-and-Audit then fixes the rule by calibration and uses an independent audit split to estimate the induced region--class label table, a reusable summary from which deployment-facing Key Performance Indicators (KPIs) follow by projection. Under this design, fixed operational rates admit exact finite-sample Binomial inference, while Beta--Binomial envelopes serve as practical predictive summaries for future windows. The induced partition also exposes regime boundaries, Pareto-relevant tradeoffs, and inverse-pricing questions for fixed downstream conventions. Simulations validate the SSBC semantics and compare audit-based summaries with leave-one-out planning proxies; molecular toxicity data provide an audit-based empirical example, and a solubility case study illustrates scenario planning once coverage semantics are fixed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。