arXiv:2601.21112cs.AIcs.SE2026-01被引 11

信息越少反而更易发现模型作恶,新方法通过筛选关键片段提升检测效果。

How does information access affect LLM monitors' ability to detect sabotage?

  • 通过提取并评估关键推理片段,实现更高效的监控。
  • 在特定场景下检测率提升16.8个百分点,优于现有方法。
  • 适合关注模型安全与资源效率的研究者使用。

前沿语言模型代理可能表现出行为错位,包括欺骗、利用奖励漏洞和追求隐藏目标。为控制潜在错位的代理,可使用大语言模型自身作为监控器。本文研究信息访问对监控性能的影响。尽管直觉认为监控器获取更多信息能更好识别异常,但实验发现当前系统常在信息较少时表现更优,这一现象称为自动化监管中的“少即是多”效应。我们在三个需隐蔽实施破坏的任务环境中验证该效应,并分析其发生条件。基于此,提出一种分层监控方法——提取与评估(EaE):由一个监控器从被监控代理的行为轨迹中提取相关片段,再由另一个监控器评分。在BigCodeBench-Sabotage任务中,使用GPT-4.1-mini作为监控器时,EaE相较最优基线检测率提升16.8个百分点。其他场景下,EaE同样表现优异或具有竞争力。此外,未意识到被监控的代理更易被捕获,且对较长推理链进行信息过滤能显著提升评分效果。成本-性能分析显示,采用信息过滤技术的Gemini与Claude模型占据大部分帕累托前沿。

原文摘要 · Abstract (English)

Frontier language model agents can exhibit misaligned behaviors, including deception, exploiting reward hacks, and pursuing hidden objectives. To control potentially misaligned agents, we can use LLMs themselves to monitor for misbehavior. In this paper, we study how information access affects LLM monitor performance. While one might expect that monitors perform better when they have access to more of the monitored agents' reasoning and actions, we find that contemporary systems often perform better with less information, a phenomenon we call the less-is-more effect for automated oversight. We demonstrate this phenomenon, and analyze the conditions under which it occurs, in three evaluation environments where agents must conduct sabotage while evading monitors. Leveraging this effect, we introduce extract-and-evaluate (EaE) monitoring -- a new hierarchical approach where one monitor isolates relevant excerpts from the monitored agent's trajectory and another monitor scores these excerpts. In BigCodeBench-Sabotage with a GPT-4.1-mini monitor, EaE improves sabotage detection rates by 16.8 percentage points over the next-best approach. In other settings, EaE either outperforms or is competitive with baselines. In addition, we find that agents unaware of being monitored can be caught much more easily and that monitors scoring longer reasoning traces gain more from information filtering. Lastly, we conduct a cost-performance analysis and find that Gemini and Claude models with monitoring techniques that involve information filtering occupy much of the Pareto frontier.

模型监控安全检测信息过滤大模型治理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。