用一致性指标评估AGI,比平均分更能发现能力短板。
A Coherence-Based Measure of AGI
- 提出连续补偿指数下的广义均值,生成面积积分度量
- 相比算术平均,该指标惩罚能力不均衡,暴露瓶颈
- 适用于认知模型和多任务基准,适合评估AGI进展
当前评估人工通用智能(AGI)的方法通常用多个认知领域表现的算术平均值来概括系统能力。这种做法隐含了‘可补偿性’假设:某些领域的优异表现可弥补其他领域的严重不足。然而真正的通用智能需要的是各核心能力的协调完备性。本文提出一种基于一致性的AGI度量方法,通过在补偿指数连续变化下计算广义均值,得到一个覆盖算术、几何与调和均值区间的面积-曲线(AUC)指标,量化系统在严格限制补偿假设时的能力鲁棒性。与算术平均不同,该指标惩罚能力不平衡,能揭示制约性能的瓶颈。我们以卡特尔-霍恩-卡罗尔(CHC)认知模型推导的认知剖面为例,展示该方法如何揭示算术平均掩盖的能力失衡;同时在17个异构基准上独立验证,证明其在更窄任务集合中仍能识别出不均衡现象。结果表明,该方法为衡量迈向AGI的进展提供了更严谨、可解释且更严格的基准。
原文摘要 · Abstract (English)
Recent approaches to evaluating Artificial General Intelligence (AGI) typically summarize a system's capability using the arithmetic mean of its proficiencies across multiple cognitive domains. While simple, this implicitly assumes compensability: exceptional performance in some areas can offset severe deficiencies in others. Genuine general intelligence, however, requires coherent sufficiency: balanced competence across all essential faculties. We introduce a coherence-based measure of AGI that integrates the generalized mean over a continuum of compensability exponents. This yields an area-under-the-curve (AUC) metric spanning arithmetic, geometric, and harmonic regimes, quantifying how robust an evaluated capability remains as compensability assumptions become stricter. Unlike the arithmetic mean, which rewards specialization, the AUC penalizes imbalance and exposes bottlenecks that constrain performance. To illustrate the framework, we apply it to cognitive profiles derived from the Cattell-Horn-Carroll (CHC) model, showing how coherence-based aggregation highlights imbalances that are obscured by arithmetic averaging. As a second, independent example, we apply the same methodology to a set of 17 heterogeneous benchmarks, demonstrating how coherence-based evaluation can reveal unevenness even in narrower task collections. These examples show that the proposed approach offers a principled, interpretable, and stricter foundation for measuring progress toward AGI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。