测试多模态大模型在无偏好场景下的随机性，发现其常出现选择僵化问题。
Evaluating Stochastic Collapse and Implicit Bias in Multimodal Large Language Models
- 设计RandomBench基准，评估模型在等效选项间是否保持均匀随机选择
- 实测发现顶级模型在随机指令下最高概率达97%，熵值降至0.068
- 该现象跨语言、跨模态普遍存在，适合关注模型公平性的研究者参考
当前多模态大模型评估主要聚焦于实用性目标，忽略了逻辑中立场景下的模型行为。在旅行推荐或日程安排等场景中，多个选项效用相当，随机性至关重要。此时确定性策略易导致重复行为，覆盖不足。为此，我们提出RandomBench基准，用于评估MLLM在等效选项间是否保持分布中性。引入RI、BCI、BII三项指标量化熵与分布偏差。实验揭示普遍存在“随机坍缩”现象：在明确随机指令下，模型无法维持均匀随机性，顶级模型（Claude Sonnet 4.6）的前1项概率高达97%（理想为25%），RI值降至0.068。大量消融实验表明，该偏差在多语言和多模态表示下均持续存在，凸显了逻辑中立决策场景中分布坍缩的顽固性。
原文摘要 · Abstract (English)
Current evaluations for Multimodal Large Language Models (MLLMs) overwhelmingly focus on utility-driven objectives, leaving model behavior under logic-neutral scenarios largely underexplored. Stochasticity is essential in scenarios where multiple actions are equally valid, such as recommending travel itineraries or daily schedules where multiple options have similar utility. In such settings, deterministic policies may lead to repetitive behaviors and reduced coverage of valid alternatives. To bridge this gap, we propose RandomBench, a benchmark designed to evaluate whether MLLMs can maintain distributionally neutral behavior when selecting among equivalent options. We further introduce three metrics, including RI, BCI, BII, to quantify entropy and distributional bias. Experiments reveal a pervasive phenomenon termed Stochastic Collapse, where MLLMs fail to maintain uniform randomness under explicit random instructions, with top-1 probabilities reaching 97% from the ideal one quarter baseline and RI dropping to 0.068 in Claude Sonnet 4.6. Extensive ablation studies further demonstrate that these deviations persist across languages and representation formats, highlighting the robustness of distributional collapse in logic-neutral decision settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。