用分而治之策略提升对不可信AI的监控安全
Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AI
- 将任务分解后由可信模型监督，不可信模型独立求解
- 监控成功率从41%提升至63%，因简化了分析环境
- 适合复杂任务场景，尤其对高阶安全控制有潜力
AI控制领域致力于开发稳健的控制协议，以防范可能故意作恶的不可信AI。然而，依赖较弱监控器检测异常行为的现有方法，在超出监控理解能力的复杂任务中常失效。本文提出基于分而治之认知的控制协议：由可信模型分解任务，不可信模型在隔离环境中求解子任务，再整合结果。该方法可通过简化监控上下文或隐藏环境漏洞来增强安全性。我们在APPS编程任务中实现并对抗性测试了基于GPT-4.1 Nano的后门攻击。结果表明：(i) 在可信监控协议中加入分而治之可使安全率从41%提升至63%；(ii) 安全性提升源于监控性能改善；(iii) 对于具备能力的大模型而言，分而治之并未增加编写后门的难度。尽管在APPS任务中实用性有限，该方法在更复杂任务中仍具前景。
原文摘要 · Abstract (English)
The field of AI Control seeks to develop robust control protocols, deployment safeguards for untrusted AI which may be intentionally subversive. However, existing protocols that rely on weaker monitors to detect unsafe behavior often fail on complex tasks beyond the monitor's comprehension. We develop control protocols based on factored cognition, in which a trusted model decomposes a task, an untrusted model solves each resultant child task in isolation, and the results are reassembled into a full solution. These protocols may improve safety by several means, such as by simplifying the context for monitors, or by obscuring vulnerabilities in the environment. We implement our protocols in the APPS coding setting and red team them against backdoor attempts from an adversarial GPT-4.1 Nano. We find that: (i) Adding factored cognition to a trusted monitoring protocol can boost safety from 41% to 63%; (ii) Safety improves because monitor performance improves; (iii) Factored cognition makes it no harder for capable LLMs to write backdoors in APPS. While our protocols show low usefulness in APPS, they hold promise for more complex tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。