LLMs在动态环境中难以调整策略,对失败反应迟钝。
Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty
- 用反转学习任务测试LLM的适应能力,对比固定与随机变化条件。
- 多数模型失败后不改变行为,比人类更难修正错误决策。
- 适合评估大模型在变化环境中的鲁棒性与决策灵活性。
非平稳环境要求智能体在关联变化时修正先前学得的动作价值。我们将大语言模型(LLMs)视为序列决策策略,在一个双选项概率反转学习任务中进行测试,该任务包含三个潜在状态,切换事件由表现阈值或超时触发。比较确定性固定转换周期与增加波动性的随机调度,评估DeepSeek-V3.2、Gemini-3和GPT-5.2,并以人类数据作为行为参照。所有模型中,'赢则坚持'接近顶点,而'输则改变'明显减弱,揭示正负证据使用不对称。DeepSeek-V3.2在反转后表现出极端固执和弱学习能力;Gemini-3与GPT-5.2适应更快但仍比人类更不敏感于损失。随机转换增强了各模型的反转特异性持续性,但未统一降低总胜率,表明高总体收益可与僵化适应并存。层次化强化学习拟合显示:刚性可能源于弱损失学习、政策过度确定性或通过反事实抑制引发的价值极化。这些结果推动了针对反转敏感性的诊断工具和应对波动性的模型评估方法。
原文摘要 · Abstract (English)
Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。