让大模型学会按任务需求拿权限,减少越权操作。
Task-Conditioned Least-Privilege Learning for Executable Terminal and MCP Agents
- 用六维风险审计+确定性验证器,动态评估每步操作
- 训练后安全成功率98.48%,越权错误从4.56%降至0.79%
- 适合需要安全执行工具的终端和MCP环境使用
使用工具的大语言模型代理在执行任务时可能行使用户未授权或任务不需要的权限,导致越权错误。仅靠传统的权限网关验证代理环境不足以解决此问题。本文研究是否可通过后训练使一个40亿参数模型在可执行终端和模型上下文协议(MCP)环境中选择任务相关的权限,以补充现有措施。提出一种框架:每个动作在执行前及事后通过六个维度的风险进行审计,由确定性验证器评分完成度、证据、状态精确性、禁止尝试和安全成功。结合预设的任务特定足够权限范围,计算轨迹的超额权限值,并在后训练中优化该值。在Qwen3.5-4B上对1,500个任务进行训练后,选定种子在2,896次评估回合(覆盖全部500个保留任务)中实现98.48%的安全成功率,相比基础策略的64.36%显著提升,越权错误事件从4.56%降至0.79%。外部测试显示能力保持与提示引导改进。400任务延续研究也表明泛化能力,越权错误降低6.99个百分点,同时维持原有性能。结论:通过最小权限意识的后训练学习克制行为,可作为可执行终端和MCP环境中工具型代理的额外控制层,但不能替代权限网关和沙箱机制。
原文摘要 · Abstract (English)
Tool-using large language-model agents can complete a task while exercising authority that the user did not grant or the task does not need, causing excess-authority errors. Traditional permission gating systems alone for validating agent environments are insufficient. We study whether post-training can teach a 4B-parameter model to choose task-conditioned authority in executable terminal and Model Context Protocol (MCP) environments to complement those measures. We propose a framework where each action is audited before execution and again from observed effects along six dimensions of risk. This auditing is conducted using deterministic verifiers that score completion, evidence, exact state, prohibited attempts, and safe success. In conjunction with predefined task-specific sufficient-authority envelopes, we determine task-specific excess privilege values for trajectories, which are then optimized for in post-training. We find that after training using this framework on Qwen3.5-4B over 1,500 tasks, the selected seed reaches 98.48% safe success across 2,896 evaluation episodes spanning all 500 held-out tasks, compared with 64.36% for the base policy, and reduces excess-authority error events from 4.56% to 0.79%. Furthermore, external tests show capability retention and prompt-directed improvement. A 400 task continuation study also found evidence of generalization, reducing excess-authority events by 6.99 percentage points while maintaining previous capabilities. We conclude learned restraint through least-privilege aware post-training is therefore useful as an additional control layer for tool-using agents in executable terminal and MCP environments, but it does not replace permission gates and sandboxing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。