arXiv:2603.27403cs.LGcs.AI2026-03被引 1

让大模型回答更可信:按问题难易动态调整答案范围,减少幻觉。

Conditional Factuality Controlled LLMs with Generalization Certificates via Conformal Sampling

  • 根据题目难度动态设定答案接受阈值,实现精准控制。
  • 在各类测试中接近目标覆盖率,且预测集比传统方法小30%以上。
  • 适合需要高可靠性输出的场景,如医疗、法律问答系统。

大语言模型需在测试时可靠控制幻觉。现有基于置信区间的办法通常仅提供边际保证,依赖单一全局阈值,导致难题覆盖不足、易题过度覆盖,预测集过大。本文提出条件事实性控制(CFC),一种后处理置信区间框架,返回带条件覆盖保证的集合输出。CFC通过在隐式‘成功’分数上进行增强分位数回归,定义连续的特征相关接受阈值,并在推理时采用固定点规则部署。理论上,我们证明在可交换性假设下,CFC满足条件覆盖;并分析其效率,证明在弱分布假设下,该条件规则比边际置信区间更样本高效。进一步推导出一种类似PAC的变体CFC-PAC,基于稳定性边界缩小名义风险水平,给出有限样本证书:条件误覆盖偏离目标最多为 $O(\sqrt{\log(1/δ)/N})$。实验显示,在合成数据、真实世界推理与问答基准及Flickr8k视觉语言模型设置中,CFC与CFC-PAC在不同难度组均稳定达到近目标覆盖率,且预测集显著小于传统置信区间和非置信区间基线。

原文摘要 · Abstract (English)

Large language models (LLMs) need reliable test-time control of hallucinations. Existing conformal methods for LLMs typically provide only \emph{marginal} guarantees and rely on a single global threshold, which can under-cover hard prompts, over-cover easy ones, and produce oversized prediction sets. We propose \emph{Conditional Factuality Control} (CFC), a post-hoc conformal framework that returns \emph{set-valued} outputs with \emph{conditional} coverage guarantees. CFC defines a continuous, feature-conditional acceptance threshold through augmented quantile regression on a latent ``success'' score, and deploys it through a fixed-point threshold rule at inference time. Theoretically, we show that CFC satisfies a conditional coverage guarantee under exchangeability and analyze its \emph{efficiency}, proving that, under mild assumptions on the score distributions, the conditional rule is strictly more sample-efficient than marginal conformal prediction at the same target coverage. We further derive a PAC-style variant, CFC-PAC, which shrinks the nominal risk level based on a stability bound, yielding a finite-sample certificate that the conditional miscoverage deviates from the target by at most $O(\sqrt{\log(1/δ)/N})$. Empirically, on synthetic data, real-world reasoning and QA benchmarks, and a Flickr8k VLM setting, CFC and CFC-PAC consistently attain near-target coverage across difficulty groups while using smaller prediction sets than CP and non-CP baselines.

大模型幻觉控制置信区间可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。