用纯语言反馈测试大模型在不确定环境下的推理能力
TextBandit: Evaluating Probabilistic Reasoning in LLMs Through Language-Only Decision Tasks
- 让大模型通过文字提示判断最优选择，无数值信息
- 通义千问3-4B选对最优选项率达89.2%，超多数模型和算法
- 为评估语言模型非数值决策能力提供新基准
大型语言模型（LLMs）在推理任务中表现日益出色，但其仅通过自然语言在不确定性环境下进行序列决策的能力仍待探索。本文提出一个新基准，让模型在多臂老虎机环境中仅通过文本反馈‘你获得了一个代币’进行交互，无法获取数值线索或显式概率，需仅凭语言线索推断潜在奖励结构并调整策略。我们评估了四个开源大模型，并与贝叶斯抽样（Thompson Sampling）、ε-贪婪、上置信界（UCB）及随机选择等经典决策算法对比。尽管多数模型表现不及基线，但通义千问3-4B达到89.2%的最优臂选择率，显著优于其他大模型和传统方法。结果表明，仅靠语言即可涌现出概率推理能力，本研究为评估自然语境下非数值决策能力迈出了关键一步。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown to be increasingly capable of performing reasoning tasks, but their ability to make sequential decisions under uncertainty only using natural language remains underexplored. We introduce a novel benchmark in which LLMs interact with multi-armed bandit environments using purely textual feedback, "you earned a token", without access to numerical cues or explicit probabilities, resulting in the model to infer latent reward structures purely off linguistic cues and to adapt accordingly. We evaluated the performance of four open-source LLMs and compare their performance to standard decision-making algorithms such as Thompson Sampling, Epsilon Greedy, Upper Confidence Bound (UCB), and random choice. While most of the LLMs underperformed compared to the baselines, Qwen3-4B, achieved the best-arm selection rate of 89.2% , which significantly outperformed both the larger LLMs and traditional methods. Our findings suggest that probabilistic reasoning is able to emerge from language alone, and we present this benchmark as a step towards evaluating decision-making capabilities in naturalistic, non-numeric contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。