提出CoVeR方法，让自回归模型生成更可靠且覆盖全面的下一个词预测。
CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Prediction
- 基于置信度预测框架，不依赖模型本身，保持搜索空间紧凑。
- 理论上保证在任意置信水平下，覆盖率达1−α以上，渐近成立。
- 适合需要高可靠性与多样化输出的场景，如医疗、金融等关键领域。
自回归预训练模型结合解码策略在复杂推理任务中表现优异。主流解码方法如束搜索虽能生成合理候选集，但缺乏可证明的覆盖保障，且难以在搜索效率与多样轨迹（尤其是长尾序列）之间取得平衡，而后者在某些实际应用中至关重要。为此，我们提出 extsc{CoVeR}，一种基于置信度预测框架的新型无模型解码策略，在维持紧凑搜索空间的同时，确保对理想轨迹具有高覆盖率。理论上，我们建立了类似 PAC 的泛化界，证明 extsc{CoVeR} 在渐近情况下对任意目标置信水平 α∈(0,1)，覆盖率不低于 1−α。
原文摘要 · Abstract (English)
Autoregressive pre-trained models combined with decoding methods have achieved impressive performance on complex reasoning tasks. While mainstream decoding strategies such as beam search can generate plausible candidate sets, they often lack provable coverage guarantees, and struggle to effectively balance search efficiency with the need for versatile trajectories, particularly those involving long-tail sequences that are essential in certain real-world applications. To address these limitations, we propose \textsc{CoVeR}, a novel model-free decoding strategy wihtin the conformal prediction framework that simultaneously maintains a compact search space and ensures high coverage probability over desirable trajectories. Theoretically, we establish a PAC-style generalization bound, guaranteeing that \textsc{CoVeR} asymptotically achieves a coverage rate of at least $1 - α$ for any target level $α\in (0,1)$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。