大模型群体共识无法提升真相判断,反而放大错误认知。
Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
- 用多轮采样聚合提升语言模型真相判断,但效果不佳。
- 25倍推理成本下,准确率仍不如单次采样,且错误更集中。
- 模型间错误高度相关,自信心评分也无法区分对错。
Pass@k等推理扩展方法在数学和代码等有外部验证的领域中能提升大模型表现,但其能否用于无可靠验证的真相判断领域?我们测试了五个基准和多个模型,发现即使将推理成本提高25倍,基于投票的聚合策略也未能持续提升准确率,反而加剧了模型间的共同误解。在不确定情况下,模型更擅长预测其他模型的回答,而非识别真实答案,表明社会预测与真相验证存在本质差异。模型错误具有强相关性,即便在分布外随机字符串上生成伪随机输出时,不同模型仍产生高度相关的结果。基于自信度加权同样无效,因为自我报告的置信度无法可靠区分正确与错误答案。这揭示了推理时扩展计算的边界:在可验证领域,额外样本有助于验证器筛选;在不可验证领域,额外样本只会强化集体错误。
原文摘要 · Abstract (English)
Pass@k and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect candidates can be filtered reliably. This raises a natural question: can we similarly scale compute to elicit gains in truthfulness for domains without convenient verification? We show that across five benchmarks and models, surprisingly, it cannot. Even at 25x the inference cost of naive sampling, polling-style aggregation yields no consistent accuracy gains over single-sample baselines and often amplifies shared misconceptions. We find that under uncertainty, models are better at predicting what other models will say within model ensembles than at identifying what is true, revealing a separation between social prediction and truth verification. Across models and benchmarks, aggregation fails to provide a robust truth signal because language model errors are strongly correlated. The source of correlation goes beyond any individual benchmark: we show that even when conditioned on out of distribution random strings and asked to produce pseudo-random outputs, different models produce correlated outputs. Confidence-based weighting provides no benefit because self-reported confidence fails to reliably distinguish correct from incorrect answers. These results delineate a boundary for inference-time scaling: in verified domains, additional samples provide more candidates for a verifier to filter; in unverified domains, additional samples merely reinforce shared misconceptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。