arXiv:2602.24080cs.AIcs.SD2026-02中稿 · ICLR被引 1

首次对语音对话系统进行图灵测试,发现当前技术仍难以以假乱真。

Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction

  • 通过2968次人类判断,评估9个主流语音对话系统的表现
  • 系统在情感表达和对话人格上表现不足,非理解能力短板
  • 提出可解释模型,用于自动评估语音系统的拟人程度

为检验现代语音到语音(S2S)系统是否具备类人对话能力,我们开展了首个针对S2S系统的图灵测试。共收集了2,968条人类对9个先进S2S系统与28名真人参与者对话的判断。结果显示,现有任何评测系统均未通过测试,揭示出显著的人类相似性差距。为诊断失败原因,我们构建了一个包含18个维度的细粒度人类相似性分类体系,并对收集的对话进行了众包标注。分析表明,瓶颈不在于语义理解,而在于副语言特征、情感表达力和对话人格。此外,我们发现现成的AI模型作为图灵测试裁判不可靠。为此,我们提出一种可解释模型,利用细粒度人类相似性评分,实现准确且透明的人机区分,成为自动评估系统拟人度的强大工具。本研究建立了首个针对S2S系统的拟人度评估框架,突破二元判定,提供深度诊断洞察,为类人对话智能的发展铺平道路。

原文摘要 · Abstract (English)

The pursuit of human-like conversational agents has long been guided by the Turing test. For modern speech-to-speech (S2S) systems, a critical yet unanswered question is whether they can converse like humans. To tackle this, we conduct the first Turing test for S2S systems, collecting 2,968 human judgments on dialogues between 9 state-of-the-art S2S systems and 28 human participants. Our results deliver a clear finding: no existing evaluated S2S system passes the test, revealing a significant gap in human-likeness. To diagnose this failure, we develop a fine-grained taxonomy of 18 human-likeness dimensions and crowd-annotate our collected dialogues accordingly. Our analysis shows that the bottleneck is not semantic understanding but stems from paralinguistic features, emotional expressivity, and conversational persona. Furthermore, we find that off-the-shelf AI models perform unreliably as Turing test judges. In response, we propose an interpretable model that leverages the fine-grained human-likeness ratings and delivers accurate and transparent human-vs-machine discrimination, offering a powerful tool for automatic human-likeness evaluation. Our work establishes the first human-likeness evaluation for S2S systems and moves beyond binary outcomes to enable detailed diagnostic insights, paving the way for human-like improvements in conversational AI systems.

语音对话图灵测试拟人度评估对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。