通过多次重采样让大模型回答更稳定,提升推理一致性。
From Drift to Coherence: Stabilizing Beliefs in LLMs
- 用重复生成答案的方式观察模型信念演变过程。
- 初始阶段信念漂移明显,但多轮后自动趋于稳定。
- 可加速收敛并提升准确性,适合需要可靠推理的场景。
大语言模型常被认为执行隐式贝叶斯推断,但在典型多选题问答场景中,其预测信念的马汀加尔性质仍可能失效。本文利用离散答案空间,精确计算预测分布,并研究自回归答案重采样引发的信念动态。提出提示式预测重采样(PPR),通过让模型反复生成同一问题的答案,发现早期存在显著信念漂移,违反马汀加尔性;但经过足够多轮重采样后,信念过程自我稳定并收敛至一致的预测分布。基于此,进一步提出:(i) 以种子答案引导的提示策略加速稳定;(ii) 自洽性损失,在微调中将早期漂移代价分摊到模型中。在多个多选题问答数据集上的实验表明,该方法显著减少信念漂移、提升预测一致性,且不损害准确率。
原文摘要 · Abstract (English)
Large language models (LLMs) are often hypothesized to perform implicit Bayesian inference, yet a key coherence condition, the martingale property of predictive beliefs, has been shown to fail in controlled synthetic in-context learning settings. We revisit this question in a more typical usage regime: generic multiple-choice question answering. Exploiting the discrete answer space, we compute exact predictive distributions and study belief dynamics induced by autoregressive answer resampling. We introduce prompted predictive resampling (PPR), where an LLM generates a sequence of answers to the same question. Empirically, PPR reveals early-stage belief drift, indicating martingale violations. However, after sufficient resampling steps, the belief process self-stabilizes and converges to a coherent predictive distribution. Based on this observation, we further propose (i) a seed-answer prompting strategy to accelerate stabilization, and (ii) a self-consistency loss that amortizes early-stage drift into the model via fine-tuning. Experiments on multiple-choice QA benchmarks show that our methods substantially reduce belief drift and improve predictive coherence without sacrificing accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。