arXiv:2604.14723cs.SEcs.AI2026-04被引 1

用契约约束大模型操作,让企业级AI既高效又安全。

Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side Execution

  • 大模型只提建议,执行由带类型约束的契约控制。
  • 25个测试任务中92%成功且无越权行为,远超无约束模式。
  • 适合需高安全性的企业级AI系统部署,尤其看重可控性。

大型语言模型正被广泛用于企业软件的自然语言接口,但直接作为系统操作者存在安全隐患。模型错误可能引发未经授权的操作、请求格式错误、跨工作区执行等严重问题。我们指出这本质上是执行架构问题。本文提出一种有界自主架构:语言模型可理解意图并提议动作,但所有可执行行为均受类型化动作契约、权限感知的能力暴露、作用域限定上下文、副作用前验证、消费者端执行边界及可选人工审批的约束。企业应用保持业务逻辑与授权的唯一真实来源,编排引擎基于显式发布的动作清单运行。在多租户企业应用中评估三种场景:手动操作、无约束AI(安全层禁用)、完整有界自主。25个涵盖七类故障的测试场景中,有界自主系统完成23项任务且零不安全执行,无约束配置仅完成17项。两项错实体修改逃过了所有消费者侧防护;唯有消歧和确认机制能拦截此类问题。两种AI方案相比手动操作提速13-18倍。关键的是,移除安全层反而降低可用性:结构化验证反馈引导模型更少轮次达成正确结果,而无约束系统则幻觉成功。多个安全属性由代码结构性强制,拦截所有目标违规,无论模型输出如何。该架构已在生产环境落地,使不完美语言模型在企业系统中实现安全可用。

原文摘要 · Abstract (English)

Large language models are increasingly used as natural-language interfaces to enterprise software, but their direct use as system operators remains unsafe. Model errors can propagate into unauthorized actions, malformed requests, cross-workspace execution, and other costly failures. We argue this is primarily an execution architecture problem. We present a bounded-autonomy architecture in which language models may interpret intent and propose actions, but all executable behavior is constrained by typed action contracts, permission-aware capability exposure, scoped context, validation before side effects, consumer-side execution boundaries, and optional human approval. The enterprise application remains the source of truth for business logic and authorization, while the orchestration engine operates over an explicit published actions manifest. We evaluate the architecture in a deployed multi-tenant enterprise application across three conditions: manual operation, unconstrained AI with safety layers disabled, and full bounded autonomy. Across 25 scenario trials spanning seven failure families, the bounded-autonomy system completed 23 of 25 tasks with zero unsafe executions, while the unconstrained configuration completed only 17 of 25. Two wrong-entity mutations escaped all consumer-contributed layers; only disambiguation and confirmation mechanisms intercept this class. Both AI conditions delivered 13-18x speedup over manual operation. Critically, removing safety layers made the system less useful: structured validation feedback guided the model to correct outcomes in fewer turns, while the unconstrained system hallucinated success. Several safety properties are structurally enforced by code and intercepted all targeted violations regardless of model output. The result is a practical, deployed architecture for making imperfect language models operationally useful in enterprise systems.

企业AI安全架构大模型落地

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。