测试公开AI系统认证，发现工具虽系统但耗时，文档缺失成瓶颈。
Practical Application and Limitations of AI Certification Catalogues in the Light of the AI Act
- 用弗劳恩霍夫认证清单系统评估模型合规性。
- 清单结构化有效，但使用耗时且依赖完整文档。
- 适合研究者与政策制定者参考认证流程缺陷。
本文在《人工智能法案》背景下，研究现有认证目录在实际应用中的表现与局限性，尝试对一个公开可用的AI系统进行认证。通过采用弗劳恩霍夫人工智能评估目录作为系统性评估工具，考察其对模型合规性的覆盖能力。结果显示，该目录虽能有效组织评估流程，但操作繁琐且耗时。我们观察到无活跃开发团队的系统存在文档不全问题，严重影响认证可行性。最后，识别出当前认证目录存在的不足，并提出优化认证流程的初步建议。
原文摘要 · Abstract (English)
In this work-in-progress, we investigate the certification of AI systems, focusing on the practical application and limitations of existing certification catalogues in the light of the AI Act by attempting to certify a publicly available AI system. We aim to evaluate how well current approaches work to effectively certify an AI system, and how publicly accessible AI systems, that might not be actively maintained or initially intended for certification, can be selected and used for a sample certification process. Our methodology involves leveraging the Fraunhofer AI Assessment Catalogue as a comprehensive tool to systematically assess an AI model's compliance with certification standards. We find that while the catalogue effectively structures the evaluation process, it can also be cumbersome and time-consuming to use. We observe the limitations of an AI system that has no active development team anymore and highlighted the importance of complete system documentation. Finally, we identify some limitations of the certification catalogues used and proposed ideas on how to streamline the certification process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。