测试大模型在看不见过去信息时,能否记住并利用历史观察做决策。
Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games

- 设计新基准,分离记忆与决策能力,模拟真实闭环控制场景。
- 最难关卡需处理128K token上下文和350张图像,现有模型仍表现不足。
- 发现模型失败主因是遗忘而非决策错误,可针对性优化训练策略。
将多模态基础模型部署为闭环策略时,常需基于已不可见的观测进行动作决策。现有基准或暴露完整状态,或混淆隐藏状态重建与其他技能,或仅在回合结束后测试回忆。本文提出RNG-Bench(重构非马尔可夫游戏)基准套件,旨在独立评估模型在多步交互中重建过往观测并据此行动的能力。该套件包含两个互补游戏:Matching Pairs要求在特定位置短暂揭示卡片身份后后续回忆;3D Maze要求将视角信息整合为空间地图。两者均在统一框架下,通过网格大小、视觉模式和观测模态三个可控难度轴进行评估。基准引入对弈协议以控制实例级方差,并提出Memory Gap指标,解耦遗忘与动作选择不佳。最难关卡每回合需处理约128K token上下文和350张图像输入,远未被前沿多模态大模型饱和。内存缺口分析表明,多数残余误差源于遗忘早期观测,而非决策不当。最后,对Qwen3.5-9B在最优策略回放与过滤模型演示数据上微调,提升RNG-Bench表现,并迁移至现有基准,且不损害通用多模态能力。
原文摘要 · Abstract (English)
Deploying multimodal foundation models as closed-loop policies increasingly requires conditioning actions on observations that are no longer visible. However, existing benchmarks either expose the full state, conflate hidden-state reconstruction with other agent skills, or test recall only after an episode has ended. We introduce RNG-Bench (Reconstructive Non-Markov Games), a benchmark suite designed to isolate a base model's ability to reconstruct past observations and act on them during multi-step interaction. RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map. Both games are evaluated under a unified harness with three controlled difficulty axes: grid size, visual pattern, and observation modality. The benchmark further introduces a head-to-head duel protocol to control for instance-level variance and a Memory Gap metric that disentangles forgetting from poor action selection. The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs. Memory Gap analysis shows that most residual errors stem from forgetting earlier observations rather than from suboptimal decision making. Finally, fine-tuning Qwen3.5-9B on optimal-policy rollouts and filtered model demonstrations improves performance on RNG-Bench and transfers to existing benchmarks without degrading general multimodal capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。