arXiv:2607.24343cs.LGcs.AI2026-07

为大模型工具调用的每个参数角色独立设风险阈值,提升安全性。

Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls

论文配图:Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls
图 1 · 摘自论文原文
  • 按参数语义角色分层设定风险阈值,避免高风险项被低风险项稀释。
  • 在六种模型和多种攻击下,实现各角色风险控制的一致性达标。
  • 适合关注大模型工具调用安全性的研究人员与开发者。

语言模型代理通过结构化工具调用执行任务,其参数携带不同风险:未经验证的内容可合理影响邮件正文,但绝不能用于设置收件人、账户、命令或凭证。现有合笼风险控制方法对整个工具调用进行认证,导致一个罕见的高风险字段失败会被周围大量良性参数平均掉,致使真正引发危害的参数未被认证。本文提出角色分层的逐字段合笼风险控制,作为校准层包裹任意逐字段检测器,为每个语义参数角色分配独立阈值与风险预算。我们证明,整体认证因粒度粗而付出代价:罕见角色的有效预算随其出现频率降低而收紧;而角色分层校准能直接对充分采样的角色提供有限样本保证,并合并最罕见的角色。在 AgentDojo 与 InjecAgent 数据集上,使用六种语言模型测试,本方法在模型迁移、攻击转移、检测器噪声、渐进漂移、未见工具集及自适应攻击等条件下,展现出最优的角色级预算合规性,可在交换性假设下或重新校准后提供正式的逐角色保障。结果表明,结构化工具调用应在语义角色层面认证,而非整体动作层面。

原文摘要 · Abstract (English)

Language-model agents act through structured tool calls whose arguments carry very different risks: untrusted content may legitimately shape an email body but should never set a recipient, account, command, or credential. Existing conformal risk control methods certify a tool call as a whole, so a failure in one rare high-risk field can be averaged away by the many benign arguments around it, leaving the argument that causes harm uncertified. We introduce role-stratified per-field conformal risk control, a calibration layer that wraps any per-field detector and assigns a separate threshold and risk budget to each semantic argument role. We show that aggregate certification pays a price of coarseness, tightening a rare role's effective budget in proportion to how often that role appears, whereas role-stratified calibration certifies each sufficiently sampled role directly with a finite-sample guarantee and pools the rarest roles. Across AgentDojo and InjecAgent with six language models, our method achieves the most consistent role-specific budget compliance among the methods we evaluate under model and attack transfer, detector noise, gradual drift, unseen tool suites, and adaptive attacks, providing formal per-role guarantees under exchangeability or after recalibration. These results suggest that structured tool calls should be certified at the semantic-role level, not the whole action.

风险控制大模型安全工具调用合笼方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。