用类型系统约束智能体行为，防止信息泄露和恶意操作。
Tracking Capabilities for Safer Agents
- 用带能力检查的Scala 3语言构建安全约束，限制工具调用权限。
- 实验表明性能几乎无损失，且能可靠阻止信息泄露和异常副作用。
- 适合关注AI安全、系统级防护的研究者与开发者。
通过工具调用与现实世界交互的智能体面临根本性安全挑战：可能泄露隐私信息、引发意外后果，或遭受提示注入攻击。为解决这些问题，我们提出在编程语言基础上构建“安全约束”：智能体不直接调用工具，而是以能力安全语言（即带有捕获检查的Scala 3）表达意图。能力是控制对效应和资源访问的程序变量。Scala的类型系统可静态追踪能力，实现细粒度控制。特别是支持局部纯性——强制子计算无副作用，从而在处理敏感数据时防止信息泄露。实验表明，利用强类型系统与能力追踪，可构建可扩展的智能体安全框架。智能体生成的能力安全代码性能基本不受影响，而类型系统能可靠阻止如信息泄露、恶意副作用等不安全行为。
原文摘要 · Abstract (English)
AI agents that interact with the real world through tool calls pose fundamental safety challenges: agents might leak private information, cause unintended side effects, or be manipulated through prompt injection. To address these challenges, we propose to put the agent in a programming-language-based "safety harness": instead of calling tools directly, agents express their intentions as code in a capability-safe language: Scala 3 with capture checking. Capabilities are program variables that regulate access to effects and resources of interest. Scala's type system tracks capabilities statically, providing fine-grained control over what an agent can do. In particular, it enables local purity, the ability to enforce that sub-computations are side-effect-free, preventing information leakage when agents process classified data. We demonstrate that extensible agent safety harnesses can be built by leveraging a strong type system with tracked capabilities. Our experiments show that agents can generate capability-safe code with no significant loss in task performance, while the type system reliably prevents unsafe behaviors such as information leakage and malicious side effects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。