arXiv:2605.29430cs.AIcs.CL2026-05被引 3

让语音识别像人一样对话纠错,通过多轮交互提升理解准确率。

Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation

论文配图:Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation
图 1 · 摘自论文原文
  • 构建闭环交互框架,结合语义修正与意图路由实现多轮纠错。
  • 在多语言和混合语种数据上,语义错误率降低幅度远超传统指标。
  • 提出新评估指标S²ER和仿真系统,适合研究人机交互的语音模型。

自动语音识别(ASR)是人机交互的核心组件,也是基于大语言模型助手的重要前端。然而,当前大多数ASR系统仍采用单次处理范式,与人类沟通中通过反复澄清和修正理解的方式不匹配,导致关键语义错误难以纠正。同时,词级指标如WER或CER无法有效反映此类问题。为此,我们首次将交互式ASR定义为多轮精炼任务,提出 extbf{Agentic ASR}——一种融合单次识别前端、语义修正、意图路由与基于推理的编辑的闭环框架。我们进一步引入 extbf{句级语义错误率}($S^2ER$),一个基于LLM的语义评估指标,并构建 extbf{交互式仿真系统}以实现可扩展、可复现的基准测试。在多语言、命名实体密集及代码切换等基准上的实验表明,多轮交互持续降低语义错误,$S^2ER$下降幅度显著高于传统词级指标。人机对齐与消融实验进一步验证了语义评判器的可靠性与框架的鲁棒性。代码已公开:https://interactiveasr.github.io/,在线演示地址:https://i-asr.sjtuxlance.com/

原文摘要 · Abstract (English)

Automatic speech recognition (ASR) is a core component of human--computer interaction and an increasingly important front-end for LLM-based assistants and agents. However, most current ASR systems still follow a single-pass paradigm, which is poorly aligned with human communication, where misunderstandings are resolved through iterative clarification and refinement. This mismatch makes it difficult to correct meaning-critical errors once they occur. Meanwhile, token-level metrics such as WER or CER cannot adequately reflect such a problem. To address these limitations, we formulate \emph{Interactive ASR} as a multi-turn refinement task and propose \textbf{Agentic ASR}, a closed-loop framework that combines a single-pass ASR front-end with semantic correction, intent routing, and reasoning-based editing. We further introduce the \textbf{Sentence-level Semantic Error Rate} ($S^2ER$), an LLM-based semantic evaluation metric, together with an \textbf{Interactive Simulation System} for scalable and reproducible benchmarking. Experiments on multilingual, named-entity-intensive, and code-switching benchmarks show that iterative interaction consistently reduces semantic errors, with much larger gains in $S^2ER$ than in conventional token-level metrics. Human--AI alignment and ablation studies further validate the reliability of the semantic judge and the robustness of the proposed framework. The code is available at: https://interactiveasr.github.io/ and the live demo is available at https://i-asr.sjtuxlance.com/

语音识别交互式系统语义纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。