用上下文学习实现快速可靠置信区间，无需重复训练。
From predictions to confidence intervals: an empirical study of conformal prediction methods for in-context learning
- 利用上下文学习在单次前向传播中生成置信区间。
- 在噪声回归任务中覆盖率达95%，优于传统方法。
- 适合需要快速不确定性估计的Transformer应用。
Transformer已成为机器学习的标准架构，展现出强大的上下文学习（ICL）能力，可在推理时通过提示完成学习。然而，针对ICL的不确定性量化仍是一个开放挑战，尤其在噪声回归任务中。本文研究了能否利用ICL实现分布无关的不确定性估计，提出一种基于分位数预测的置信区间构造方法。传统分位数方法因需反复拟合模型而计算成本高，我们借助ICL特性，在单次前向传播中高效生成置信区间。实证分析表明，结合上下文学习的分位数预测（CP with ICL）在多种场景下均能实现稳健且可扩展的不确定性估计。此外，我们在分布偏移下评估了其性能，并建立了指导模型训练的缩放规律。该工作连接了上下文学习与分位数预测，为基于Transformer模型的不确定性量化提供了理论严谨的新框架。
原文摘要 · Abstract (English)
Transformers have become a standard architecture in machine learning, demonstrating strong in-context learning (ICL) abilities that allow them to learn from the prompt at inference time. However, uncertainty quantification for ICL remains an open challenge, particularly in noisy regression tasks. This paper investigates whether ICL can be leveraged for distribution-free uncertainty estimation, proposing a method based on conformal prediction to construct prediction intervals with guaranteed coverage. While traditional conformal methods are computationally expensive due to repeated model fitting, we exploit ICL to efficiently generate confidence intervals in a single forward pass. Our empirical analysis compares this approach against ridge regression-based conformal methods, showing that conformal prediction with in-context learning (CP with ICL) achieves robust and scalable uncertainty estimates. Additionally, we evaluate its performance under distribution shifts and establish scaling laws to guide model training. These findings bridge ICL and conformal prediction, providing a theoretically grounded and new framework for uncertainty quantification in transformer-based models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。