用群体智慧提升大模型估算能力，效果显著优于传统方法。
Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding
- 让模型多次生成答案后取中位数，模拟群体智慧
- 在多个真实场景中，中位数法比贪婪解码准确率更高
- 适合需要快速估算的现实任务，如预测与物理推理
估算——对物体或事件进行近似定量判断——是常见现实技能，但在大语言模型研究中仍被忽视。本文构建了三个估算数据集：MARBLES（物理估算，如杯中能装多少颗弹珠）、FUTURE（未来预测，如2024年美国总统选举结果）和ELECPRED（电力预测）。受‘群体智慧’（Wisdom of Crowds, WOC）启发，我们提出一种针对大模型的WOC解码策略：通过多次采样生成答案后取中位数。实验在人类参与者中复现了群体智慧效应，并发现大模型同样受益于该策略：中位数聚合在多种任务上持续优于贪婪解码、自一致性解码和均值解码。这表明大模型具备支持近似推理的世界模型。研究将估算视为探测大模型世界知识的有效工具，并验证了WOC解码在真实世界任务中的性能提升潜力。
原文摘要 · Abstract (English)
Guesstimation -- the task of making approximate quantitative estimates about objects or events -- is a common real-world skill, yet remains underexplored in large language model (LLM) research. We introduce three guesstimation datasets: MARBLES, FUTURE, and ELECPRED, spanning physical estimation (e.g., how many marbles fit in a cup) to abstract predictions (e.g., the 2024 U.S. presidential election). Inspired by the social science concept of Wisdom of Crowds (WOC)- where the median of multiple estimates improves accuracy-we propose WOC decoding for LLMs. We replicate WOC effects in human participants and find that LLMs exhibit similar benefits: median aggregation across sampled responses consistently improves accuracy over greedy decoding, self-consistency decoding, and mean decoding. This suggests that LLMs encode a world model that supports approximate reasoning. Our results position guesstimation as a useful probe of LLM world knowledge and highlight WOC decoding as a strategy for enhancing LLM guesstimation performance on real-world tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。