用强化学习动态调整模型拒答阈值,提升大模型在安全场景下的可靠性。
Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models
- 将置信度阈值设为可学习策略,通过强化学习动态优化拒答决策
- 在多个数据集上实现90%覆盖率下准确率提升3.2%,幻觉检测AUROC提高22.19%
- 适用于需要高可靠性的医疗、金融等安全关键场景
大型语言与视觉语言模型(LLMs/VLMs)在安全关键应用中日益普及,但其决策过程不透明,难以评估风险与可靠性。不确定性量化(UQ)可评估预测置信度并实现高不确定性时的拒答。共形预测(CP)是主流的UQ方法,提供统计保证,但依赖静态阈值,无法适应任务复杂度和数据分布变化,导致准确率、覆盖率与信息量之间的权衡不佳。为此,我们提出可学习的共形拒答策略,将强化学习(RL)与CP结合,动态优化拒答阈值。将CP阈值视为可调节动作,该方法在最小化预测集大小的同时保持可靠覆盖率。在多种LLM/VLM基准上的广泛评估表明,本方法优于最小模糊分类器(LAC)和自适应预测集(APS),准确率最高提升3.2%,幻觉检测的AUROC提升22.19%,不确定性引导的选代生成(AUARC)提升21.17%,校准误差降低70%-85%。上述优势在多个模型与数据集上均成立,且始终满足90%覆盖率目标,验证了该方法在安全关键应用中更有效、更灵活的可靠性决策能力。代码已开源:https://github.com/sinatayebati/vlm-uncertainty。
原文摘要 · Abstract (English)
Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability. Uncertainty quantification (UQ) helps assess prediction confidence and enables abstention when uncertainty is high. Conformal prediction (CP), a leading UQ method, provides statistical guarantees but relies on static thresholds, which fail to adapt to task complexity and evolving data distributions, leading to suboptimal trade-offs in accuracy, coverage, and informativeness. To address this, we propose learnable conformal abstention, integrating reinforcement learning (RL) with CP to optimize abstention thresholds dynamically. By treating CP thresholds as adaptive actions, our approach balances multiple objectives, minimizing prediction set size while maintaining reliable coverage. Extensive evaluations across diverse LLM/VLM benchmarks show our method outperforms Least Ambiguous Classifiers (LAC) and Adaptive Prediction Sets (APS), improving accuracy by up to 3.2%, boosting AUROC for hallucination detection by 22.19%, enhancing uncertainty-guided selective generation (AUARC) by 21.17%, and reducing calibration error by 70%-85%. These improvements hold across multiple models and datasets while consistently meeting the 90% coverage target, establishing our approach as a more effective and flexible solution for reliable decision-making in safety-critical applications. The code is available at: {https://github.com/sinatayebati/vlm-uncertainty}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。