为安全关键场景的AI系统提供可验证的全周期认证框架
Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned
- 基于欧盟人工智能法案构建可测试的审计清单
- 通过统计性能要求与独立数据验证模型可信度
- 适合监管机构、开发者及合规审查人员参考
人工智能在安全关键领域应用日益广泛,但实际可用的安全性、合法性与社会可接受性认证方案仍稀缺。本文介绍由TÜV AUSTRIA开发的可信AI框架,该框架是一个自2019年起持续演进的端到端审计目录与方法论,旨在评估和认证机器学习系统。其以安全软件开发、功能需求、伦理与数据隐私三大支柱为基础,将欧盟人工智能法案的高层义务转化为具体可测标准。核心概念是‘功能可信性’,即结合统计定义的应用域、基于风险的最低性能要求以及对独立采样数据的统计测试,从而在真实场景中提供透明且可复现的模型质量证据。我们概述了评估的功能需求,涵盖AI系统全生命周期。同时分享实践中的经验教训,包括数据泄露、领域定义不清、偏见忽视及分布漂移控制缺失等常见问题。还讨论了鲁棒性、算法公平性及认证后要求等关键议题,提出当前结论与未来研究路线图。总体而言,该方法通过衔接技术最佳实践与欧洲新兴标准,为监管者、提供商与用户提供了合法合规、功能可信且可认证的AI系统实施路径。
原文摘要 · Abstract (English)
There is an increasing adoption of artificial intelligence in safety-critical applications, yet practical schemes for certifying that AI systems are safe, lawful and socially acceptable remain scarce. This white paper presents the TÜV AUSTRIA Trusted AI framework an end-to-end audit catalog and methodology for assessing and certifying machine learning systems. The audit catalog has been in continuous development since 2019 in an ongoing collaboration with scientific partners. Building on three pillars - Secure Software Development, Functional Requirements, and Ethics & Data Privacy - the catalog translates the high-level obligations of the EU AI Act into specific, testable criteria. Its core concept of functional trustworthiness couples a statistically defined application domain with risk-based minimum performance requirements and statistical testing on independently sampled data, providing transparent and reproducible evidence of model quality in real-world settings. We provide an overview of the functional requirements that we assess, which are oriented on the lifecycle of an AI system. In addition, we share some lessons learned from the practical application of the audit catalog, highlighting common pitfalls we encountered, such as data leakage scenarios, inadequate domain definitions, neglect of biases, or a lack of distribution drift controls. We further discuss key aspects of certifying AI systems, such as robustness, algorithmic fairness, or post-certification requirements, outlining both our current conclusions and a roadmap for future research. In general, by aligning technical best practices with emerging European standards, the approach offers regulators, providers, and users a practical roadmap for legally compliant, functionally trustworthy, and certifiable AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。