用可解释的强模型监控弱模型，防止串通作弊。
Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agents
- 让强模型带思维链，由弱模型审查其推理过程
- 在串通攻击下仍能提升检测率，需原始思维链数据
- 适合需要长期可靠控制的前沿AI系统
可信监控是人工智能控制的核心。然而，随着前沿模型能力提升，可信与不可信模型之间的能力差距可能使可信模型无法有效监督。我们提出“自举监控”协议：在监督链中引入一个具有透明思维链推理的更强、中间级不可信模型（U_m），由该模型评估智能体行为，再由较弱的可信模型（T）监督其推理过程以发现串通。我们在多轮软件工程任务（BashArena）上对多个智能体和监控器进行了评估。结果显示，即使不可信监控者主动与智能体串通，只要拥有其原始思维链，自举监控也能显著提升检出率，优于仅依赖可信模型的监控方式。结果表明，该方法可延长可信模型在先进AI环境下的可用寿命。
原文摘要 · Abstract (English)
Trusted monitoring is a cornerstone of AI control. However, as frontier models grow more capable, the increasing capabilities gap between trusted and untrusted models may render trusted models unreliable monitors. We introduce \emph{bootstrapped monitoring}, a protocol that addresses this by inserting a stronger, intermediate untrusted model with transparent chain-of-thought reasoning into the oversight chain. The untrusted monitor ($U_m$) evaluates the agent's actions, while a weaker trusted model ($T$) oversees $U_m$'s reasoning to detect collusion. We evaluate bootstrapped monitoring on multi-turn software engineering tasks (BashArena) across multiple agents and monitors. Bootstrapped monitoring substantially improves catch rates over trusted-only monitoring, even when the untrusted monitor actively colludes with the agent, provided we have access to its raw chain-of-thought. Our results suggest that bootstrapped monitoring can extend the useful lifetime of trusted models in control as AI capabilities advance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。