让AI像人一样边用边学,测试时自我改进决策能力。
Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning
- 构建双模块结构:元推理与动作推理协同,动态存储规则和经验。
- 在12个未见游戏上达成9项最高分,显著优于基线模型。
- 适合研究自适应智能体、持续学习的学者与工程师。
近期视觉-语言模型具备强大的感知推理能力,但在测试时遇到新任务往往难以高效适应。人类则依靠带有记忆的元认知机制,通过元认知控制持续优化策略。为此,我们提出元认知测试时推理(MCTR)框架,使模型在测试阶段具备学习、适应和自我提升的能力。MCTR借鉴人类元认知的双层结构,包含元推理模块与对象推理模块,各自配备专用记忆系统,实现分层自适应推理。元推理模块通过测试时观察,逐步构建结构化记忆,存储任务相关规则、环境模式及行动-结果关系;动作推理模块则基于上下文感知和策略推理,动态检索并整合记忆知识以决定最优动作。该模块通过提出的元认知测试时强化学习持续更新策略,随知识记忆演进而自适应。我们在45个Atari游戏中评估(33个已见,12个未见),MCTR在未见游戏上取得9项最高分,表现优于基线。消融分析、学习动态与案例研究揭示两模块互补性,并表明元推理正向类人适应策略演化。
原文摘要 · Abstract (English)
Recent Vision-Language Models (VLMs) exhibit strong perceptual reasoning abilities, yet they often struggle to adapt efficiently when encountering novel tasks at test time. In contrast, humans leverage the metacognitive model with memory, enabling continuous strategy refinement through metacognitive control when faced with new challenges. To bridge this gap, we propose metacognitive test-time reasoning (MCTR), a framework that equips models with the ability to learn, adapt, and improve during test time through metacognitive self-updating. Inspired by the dual structure of human metacognition, MCTR comprises meta-level and object-level VLM reasoning modules, each equipped with dedicated memory systems for hierarchical adaptive reasoning. Specifically, MCTR consists of (1) a meta-reasoning module which incrementally builds a structured memory by discovering and storing task-relevant rules, environmental patterns, and action-outcome relationships from test-time observations as natural language descriptions; and (2) an action-reasoning module that determines optimal actions through context-aware perception and strategic reasoning by dynamically retrieving and integrating knowledge from memory. The action-reasoning module continuously updates its policy through proposed metacognitive test-time reinforcement learning, adapting as knowledge memory evolves. We evaluate MCTR on 45 Atari games (33 seen, 12 unseen). MCTR demonstrates robust test-time adaptation, achieving 9/12 top-1 results on unseen games compared with baselines. Analyses through ablations, learning dynamics, and case studies reveal the complementary contributions of both components and show meta-reasoning evolving toward human-like adaptation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。