用不可区分性定义智能比较,让模型互评高低。
The Generalized Turing Test: A Foundation for Comparing Intelligence

- 以对方能否分清真假来判定智能强弱,不依赖特定任务。
- 实测数千次对比,结果呈现层级结构,符合已有认知。
- 适合研究智能评估与训练目标的通用化设计者。
我们提出广义图灵测试(GTT),一种通过不可区分性比较任意智能体能力的形式化框架。当智能体B作为判别者无法可靠区分与智能体A(被指令模仿B)的交互和与另一实例B的交互时,定义A ≥ B。该框架不依赖数据集或任务,具有相对智能的抽象意义。我们研究了其结构特性,包括传递性条件及等价类排序的可能性,并引入带查询、有限交互和固定判别者的变体。为验证理论,我们在多个现代模型上实例化该框架,通过数千次试验评估成对不可区分性。结果呈现分层结构,与现有排名一致,表明该框架可产生有意义的实证排序。研究认为不可区分性是理解智能的统一视角,为评估乃至训练目标提供脱离固定基准的潜在基础。
原文摘要 · Abstract (English)
We introduce the Generalized Turing Test (GTT), a formal framework for comparing the capabilities of arbitrary agents via indistinguishability. For agents A and B, we define the Turing comparator A $\geq$ B to hold if B, acting as a distinguisher, cannot reliably distinguish between interactions with A (instructed to imitate B) and another instance of B. This yields a dataset- and task-agnostic notion of relative intelligence. We study the comparator's structure, including conditions under which it is transitive and therefore induces an ordering over equivalence classes, and we define and analyze variants with querying, bounded interaction, and fixed distinguishers. To complement the theory, we instantiate the framework on a collection of modern models, empirically evaluating pairwise indistinguishability across thousands of trials. The resulting comparisons exhibit a stratified structure consistent with existing rankings, hinting that the proposed framework yields meaningful empirical orderings. Our results position indistinguishability as a unifying lens for reasoning about intelligence, suggesting a foundation for evaluation and, potentially, training objectives that are inherently independent of fixed datasets or benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。