提出新方法量化长文本生成中的不确定性，提升事实准确性
IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation
- 通过提问-回应范式，结合样本间一致性和样本内忠实性评估不确定性
- 在两个主流长文本数据集上表现优于现有方法，小模型效果提升显著
- 适合需要高可信度生成的场景，如医疗、法律等专业领域
尽管大语言模型（LLM）发展迅速，但其生成结果的不确定性量化仍是难题。现有方法多限于短文本或受限输出，而真实应用常需长篇自由文本生成。此时模型虽语义连贯，却可能事实错误，且语义复杂、结构多样。为此，本文提出交互式不确定性量化（IUQ）框架，利用样本间一致性与样本内忠实性，对长文本生成进行不确定性评估。通过‘提问-回应’范式，可提供声明级别的不确定性度量及模型忠实度。在多个模型家族与规模下测试，结果表明IUQ在两个主流长文本生成数据集上均优于现有方法。代码已开源：https://github.com/louisfanhz/IUQ。
原文摘要 · Abstract (English)
Despite the rapid advancement of Large Language Models (LLMs), uncertainty quantification in LLM generation is a persistent challenge. Although recent approaches have achieved strong performance by restricting LLMs to produce short or constrained answer sets, many real-world applications require long-form and free-form text generation. A key difficulty in this setting is that LLMs often produce responses that are semantically coherent yet factually inaccurate, while the underlying semantics are multifaceted and the linguistic structure is complex. To tackle this challenge, this paper introduces Interrogative Uncertainty Quantification (IUQ), a novel framework that leverages inter-sample consistency and intra-sample faithfulness to quantify the uncertainty in long-form LLM outputs. By utilizing an interrogate-then-respond paradigm, our method provides reliable measures of claim-level uncertainty and the model's faithfulness. Experimental results across diverse model families and model sizes demonstrate the superior performance of IUQ over two widely used long-form generation datasets. The code is available at https://github.com/louisfanhz/IUQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。