arXiv:2604.05229cs.AIcs.HC2026-04被引 6

将治理标准转化为可执行的运行时防护机制,解决智能体系统执行中的风险管控难题。

From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI

  • 提出分层翻译方法,将治理目标映射到设计约束、运行时干预等四类控制层。
  • 明确运行时防护需满足可观测、确定性、时效性,仅对关键环节实施动态干预。
  • 通过采购代理案例验证方法有效性,适合关注AI安全与合规的工程团队。

智能体型AI系统具备规划、使用工具、维护状态及产生多步外部影响的能力,其特性导致治理风险主要出现在执行过程中,而非模型开发或部署阶段。因此,ISO/IEC 42001、ISO/IEC 23894、ISO/IEC 42005、ISO/IEC 5338、ISO/IEC 38507以及NIST AI风险管理框架等治理标准对这类系统尤为相关,但无法直接生成可执行的运行时防护机制。本文提出一种分层翻译方法,将标准导出的治理目标映射至四个控制层级:治理目标、设计期约束、运行时中介与保障反馈。该方法区分了治理目标、技术控制、运行时防护与保障证据,引入控制元组和运行时可执行性评估标准以实现层级分配,并在采购代理案例中加以演示。核心主张为:治理标准应指导控制在架构、运行时策略、人工升级与审计中的布局,而运行时防护仅适用于可观测、确定且具有时效性的控制措施。

原文摘要 · Abstract (English)

Agentic AI systems plan, use tools, maintain state, and produce multi-step trajectories with external effects. Those properties create a governance problem that differs materially from single-turn generative AI: important risks emerge dur- ing execution, not only at model development or deployment time. Governance standards such as ISO/IEC 42001, ISO/IEC 23894, ISO/IEC 42005, ISO/IEC 5338, ISO/IEC 38507, and the NIST AI Risk Management Framework are therefore highly relevant to agentic AI, but they do not by themselves yield implementable runtime guardrails. This paper proposes a layered translation method that connects standards-derived governance objectives to four control layers: governance objectives, design- time constraints, runtime mediation, and assurance feedback. It distinguishes governance objectives, technical controls, runtime guardrails, and assurance evidence; introduces a control tuple and runtime-enforceability rubric for layer assignment; and demonstrates the method in a procurement-agent case study. The central claim is modest: standards should guide control placement across architecture, runtime policy, human escalation, and audit, while runtime guardrails are reserved for controls that are observable, determinate, and time-sensitive enough to justify execution-time intervention.

AI治理运行时防护智能体系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。