arXiv:2605.14186cs.LG2026-05被引 1

让大模型学会自我判断并据此调整推理,提升准确率。

LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling

论文配图:LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling
图 1 · 摘自论文原文
  • 分离自我监控与推理过程,用认知信号控制思考
  • 在多个基准上将准确率从48.3提升至56.9
  • 无需调参或微调,适合想提升推理能力的开发者

大型语言模型(LLMs)常表现出自我监控的有用信号:在解决问题前可预估成功率,解决后可判断答案正确性。然而这些信号通常仅被单独测量,未用于控制推理过程。本文受认知心理学中Nelson-Narens理论启发,提出一种元认知引导框架,将监控与推理分离。对每个问题,模型先报告求解前的‘知道感’(FOK),每次尝试后报告‘学习判断’(JOL)。不将这些信号视为被动置信度,而是作为显式推理控制接口:决定何时信任当前答案、何时以简短反馈重试,何时将多次尝试交由聚合器处理。在文本、代码和多模态推理基准上,该框架显著提升了固定版本Claude Sonnet-4.6模型的表现,未进行参数更新或任务特定微调。在公开基准快照中,综合准确率从48.3提升至56.9,并超越了三项主要评估设置(HLE-Verified、LiveCodeBench v6、R-Bench-V)的最强榜单条目。结果表明,强模型已具备潜在元认知能力,但需显式控制框架才能发挥作用。

原文摘要 · Abstract (English)

Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succeed, and after solving it, they can judge whether their answer is likely to be correct. However, these signals are typically measured or elicited in isolation, rather than used to control inference. In this work, we ask whether LLMs possess latent metacognitive ability that can be turned into effective test-time control. Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning. For each problem, the model first reports a pre-solve feeling-of-knowing (FOK) signal; after each solve attempt, it reports a post-solve judgment-of-learning (JOL) signal. Rather than treating these signals as passive confidence estimates, the harness turns them into an explicit control interface for reasoning: it decides when to trust the current solution, when to retry with compact metacognitive feedback, and when to pass multiple attempts to a final aggregator. Across text, code, and multimodal reasoning benchmarks, our harness substantially improves a fixed Claude Sonnet-4.6 base model without parameter updates or benchmark-specific fine-tuning. On the evaluated public benchmark snapshots, it raises pooled accuracy from 48.3 to 56.9 and exceeds the strongest listed leaderboard entries on the three primary evaluation settings: HLE-Verified, LiveCodeBench v6, and R-Bench-V. These results suggest that strong LLMs may already possess useful metacognitive ability, but require an explicit control harness to act on it during reasoning.

元认知推理增强大模型测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。