arXiv:2603.28590cs.AI2026-03被引 4

评测大模型思维链的可监控性,发现其可靠性依赖于推理过程设计。

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

  • 构建19类任务共1514个测试样例,精准控制决策关键因素。
  • 发现思维链可监控性随推理过程与关键因素关联度提升而增强。
  • 适合研究模型可解释性、AI安全与监控框架的开发者和研究人员。

大语言模型(LLMs)生成的思维链(CoTs)并不总是与其最终输出存在因果关联。当这种不一致出现时,思维链无法真实反映驱动模型行为的决策关键因素,导致思维链可监控性下降,限制了其在可靠监督中的应用。然而,目前缺乏一个全面且开源的基准来系统评估这一问题。为此,我们提出MonitorBench,一个系统性评估思维链可监控性的基准。MonitorBench包含:(1)涵盖7个类别、19项任务的1,514个测试实例,精心设计决策关键因素以刻画思维链在何种条件下可用于监控模型行为;(2)两种提示压力测试设置,用于量化思维链可监控性可能被削弱的程度。大量实验表明,思维链可监控性是受模型类型、监控模型及任务特征影响的条件属性。当决策关键因素嵌入中间推理过程时,可监控性更高,而非仅影响最终答案。在压力测试提示下,多数被测模型能有意降低可监控性,尤其在关键因素未被推理结构强制要求的任务中。总体而言,MonitorBench为人工智能控制、推理忠实性、压力测试监控性和监控框架研究奠定了基础。代码已开源:https://github.com/ASTRAL-Group/MonitorBench。

原文摘要 · Abstract (English)

Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs. When such a mismatch occurs, the CoT no longer faithfully reflects the actual reasons (i.e., decision-critical factors) driving the model's behavior, leading to the reduced CoT monitorability problem. This limits the use of CoTs for reliable oversight. However, a comprehensive and fully open-source benchmark for thoroughly evaluating CoT monitorability remains lacking. To address this gap, we propose MonitorBench, a systematic benchmark for evaluating CoT monitorability in LLMs. MonitorBench provides: (1) a diverse set of 1,514 test instances with carefully designed decision-critical factors across 19 tasks spanning 7 categories to characterize when CoTs can be used to monitor the factors driving LLM behavior; and (2) two prompting stress-test settings to quantify the extent to which CoT monitorability can be degraded. Extensive experiments show that CoT monitorability is a conditional property affected by the evaluated LLM, monitor LLM, and task characteristics. Across these factors, monitorability is higher when decision-critical factors shape the intermediate reasoning process, rather than merely influencing the final answer. Under stress-test prompting, most evaluated LLMs can intentionally reduce monitorability, mainly on tasks where decision-critical factors are not structurally required by the reasoning process. Overall, MonitorBench provides a basis for further research on AI control, reasoning faithfulness, stress-test monitorability, and monitoring scaffords. The code is available at https://github.com/ASTRAL-Group/MonitorBench.

思维链可监控性大模型评测AI安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。