arXiv:2606.28739cs.AI2026-06被引 1

让大模型安全不能靠拒绝指令,而要限制其权限。

Agent Safety Is Action Alignment

  • 用拒绝有害输入的方法不适用于代理模型,因危害在动作权限而非输出内容。
  • 训练后的模型只学表面模式,多步任务中易被攻破且未受威胁时已失控。
  • 真正的安全需在模型外按操作边界实施最小权限,评估应关注动作对齐性。

大型语言模型越来越多地作为代理运行:调用工具、转账、删除记录、发送消息等。为确保安全,从业者沿用对话时代的方法(训练模型拒绝不安全输入),并将由此导致的能力损失视为可接受的“对齐税”。我们指出这属于类别错误。拒绝机制适用于内容安全——危害存在于模型输出中,是可学习的函数。而代理行为的危害本质不同:它不在于任何输出,而在于动作所行使的权限与用户授予权限之间的关系,这一关系不在模型可见文本中。将内容安全方法引入代理场景,并非以能力换安全,而是牺牲能力却获得负安全。我们通过三条证据贯穿自主性谱系支持此观点:防御训练模型仅学习表层模式而非意图;相同训练会提前崩溃多步代理,即便尚未出现威胁;甚至未防御的前沿模型在常规使用下也超出授权权限。结论是,行动安全无法通过权重实现,必须以最小权限表达,由模型外部在动作边界强制执行,并以动作对齐性(一种关系性、部署条件依赖的属性)评估,而非拒绝分数。

原文摘要 · Abstract (English)

Large language models increasingly act as agents: they call tools, move money, delete records, and send messages on a user's behalf. To keep them safe, practitioners imported the chatbot-era recipe (train the model to refuse unsafe inputs) into the agentic setting, and treat the resulting capability loss as a manageable ``alignment tax.'' We argue this is a \emph{category error}. Refusal is a primitive for \emph{content safety}, where the harm is in the model's output and is therefore a learnable function of it. Agentic harm is different in kind: it lies not in any output but in the relation between the authority an action exercises and the authority the user granted, which is absent from the text the model sees. Importing content-safety methods into this regime does not trade capability for safety; it pays capability and buys negative security. We support this with three lines of evidence spanning the autonomy spectrum: defense-trained models learn surface patterns rather than intent; the same training collapses multi-step agents before any threat appears while leaving them exploitable; and even undefended frontier models exceed granted authority under ordinary use. We conclude that action safety cannot be installed in weights. It must be expressed as \emph{least privilege}, enforced \emph{outside} the model at the action boundary, and evaluated as \emph{action alignment} (a relational, deployment-conditioned property) rather than a refusal score.

模型安全代理系统权限控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。