让AI说话更可信：用检索增强重写，提升语言自信表达的准确性
Retrieval-Augmented Linguistic Calibration
- 将语言自信建模为概率分布，捕捉理解差异
- 引入信度分歧（FD）量化真相揭露时观众信念的意外程度
- 轻量级后处理框架，适配多模型与问答任务
语言线索如“我认为”和“可能”提供了直观的置信度表达方式，但通用且有原则的语义置信度校准框架仍不完善。共现语言线索、上下文变化及主观受众解读带来独特挑战。为此，我们将语言置信度建模为陈述正确性可能被感知的概率分布，捕捉解释变异性，而传统标量表示会忽略这一点。在此分布框架下，我们引入信度作为补充评估维度，并提出信度分歧（FD），一种基于信息论的度量，用于量化真相揭示时观众信念产生的意外程度。基于此，我们提出检索增强的语言校准（RALC），一种轻量级后处理流水线，通过检索增强重写将校准后的置信信号回传至自然语言。在三个QA基准和五个LLM家族上，RALC将域内信度和校准度分别提升最高达66%和58%，优于黑箱与灰箱校准基线。
原文摘要 · Abstract (English)
Linguistic cues such as "I believe" and "probably" offer an intuitive interface for communicating confidence, yet a generalisable, principled calibration framework for linguistic confidence expressions remains underexplored. In particular, co-occurring linguistic cues, contextual variation, and subjective audience interpretation pose unique challenges. We therefore model linguistic confidence as a distribution over plausible perceived probability values that a statement is correct, capturing interpretation variability that scalar representations discard. Within this distributional framework, we introduce faithfulness as a complementary evaluation dimension and present Faithfulness Divergence (FD), an information-theoretic metric quantifying the surprise induced in audience beliefs upon truth revelation. Building on these foundations, we present Retrieval-Augmented Linguistic Calibration (RALC), a lightweight post-hoc pipeline that propagates calibrated confidence signals back into natural language via retrieval-augmented rewriting. Across three QA benchmarks and five LLM families, RALC improves in-domain faithfulness and calibration up to 66% and 58%, respectively, outperforming black-box and grey-box calibration baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。