测试大模型能否无记忆地策划绕过安全协议，发现当前模型能力有限。
Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?
- 设计8个环境与6类能力隔离机制，评估模型无记忆状态下的策略谋划能力。
- 所有测试模型（含Claude 3.5和OpenAI推理模型）均未展现显著绕过能力。
- 若允许跨上下文共享计划，性能明显提升，提示可缓解最坏假设。
AI控制协议旨在部署人工智能系统时防止其造成不可接受后果。本文研究大模型在无记忆状态下（上下文间无共享记忆）自主生成并执行绕过控制协议策略的能力。模型需在每个上下文中可靠生成最优计划，以校准概率采取行动，并在不通信的情况下协调自身多个实例的策略。为此，我们构建了包含八种环境的Subversion Strategy Eval评估套件，涵盖多种协议与战略能力，并设置六组功能模块以隔离个体能力。评估在Inspect-AI中实现并开源。测试对象包括Claude 3.5系列（含仅助人类）及OpenAI推理模型。结果表明，所有模型在无记忆条件下均未表现出显著的绕过能力。但若赋予模型跨上下文共享计划的能力，性能显著提升。我们希望该评估能作为模型具备绕过能力的早期预警指标，并放松对模型拥有完美战略能力的最坏假设。
原文摘要 · Abstract (English)
An AI control protocol is a plan for usefully deploying AI systems that aims to prevent an AI from intentionally causing some unacceptable outcome. This paper investigates how well AI systems can generate and act on their own strategies for subverting control protocols whilst operating statelessly (without shared memory between contexts). To do this, an AI system may need to reliably generate optimal plans in each context, take actions with well-calibrated probabilities, and coordinate plans with other instances of itself without communicating. We develop Subversion Strategy Eval, a suite of eight environments, covering a range of protocols and strategic capabilities, and six sets of affordances that help isolate individual capabilities. We implement the evaluation in Inspect-AI and release it open-source. We evaluate Claude 3.5 models, including helpful-only versions, as well as OpenAI reasoning models. None of the models demonstrate substantial capability in strategizing to subvert control protocols statelessly. However, providing models with additional affordances, such as the ability to share a plan between contexts, can substantially improve performance. We hope our evaluations can act as a leading indicator for when models are capable of subverting control protocols and also relax the worst-case assumption of perfect strategic ability in AI control evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。