构建AI能力评估的通用框架，提升评估透明度与可比性。
A Conceptual Framework for AI Capability Evaluations
- 提出一套结构化分析方法，梳理现有评估体系
- 支持跨评估对比，揭示方法论薄弱环节
- 适合研究者、从业者和政策制定者使用
随着AI系统日益融入社会，设计良好且透明的评估已成为人工智能治理的关键工具，为决策提供关于系统能力与风险的证据。然而，如何全面而可靠地开展这些评估仍缺乏清晰指引。为此，我们提出一个概念框架，用于分析AI能力评估，采用结构化、描述性的方法，系统化梳理广泛使用的评估方法与术语，不强制新分类或固定格式。该框架有助于提升评估的透明度、可比性和可解释性，使研究者能识别方法缺陷，实践者可优化评估设计，政策制定者则获得可操作的工具，以审视、比较和理解复杂的评估环境。
原文摘要 · Abstract (English)
As AI systems advance and integrate into society, well-designed and transparent evaluations are becoming essential tools in AI governance, informing decisions by providing evidence about system capabilities and risks. Yet there remains a lack of clarity on how to perform these assessments both comprehensively and reliably. To address this gap, we propose a conceptual framework for analyzing AI capability evaluations, offering a structured, descriptive approach that systematizes the analysis of widely used methods and terminology without imposing new taxonomies or rigid formats. This framework supports transparency, comparability, and interpretability across diverse evaluations. It also enables researchers to identify methodological weaknesses, assists practitioners in designing evaluations, and provides policymakers with an accessible tool to scrutinize, compare, and navigate complex evaluation landscapes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。