arXiv:2506.18532cs.CLcs.LG2025-06被引 1

端到端语音语法纠错,提升二语学习反馈精准度。

End-to-End Spoken Grammatical Error Correction

  • 构建基于Whisper的端到端框架,避免多模块误差累积。
  • 通过伪标签扩充数据至2500小时,解决语音标注数据稀缺问题。
  • 提出参考对齐与置信度评估,精准识别并过滤错误修正项。

语法纠错(GEC)和反馈在支持第二语言学习者、教育工作者及考官方面至关重要。尽管书面GEC已成熟,但语音GEC(SGEC)因话语不流畅、转录错误及输入无结构而更具挑战性。传统系统采用自动语音识别(ASR)、不流畅检测和GEC的级联流程,易产生误差传播。本文研究基于Whisper基础模型的端到端(E2E)SGEC框架,比较级联、部分级联与全端到端架构。针对E2E系统中缺乏带标注的语音数据问题,引入自动伪标签框架,将训练数据从77小时扩展至超过2500小时。为提升准确性,利用ASR输出引入上下文信息。关键在于生成学习者错误的候选反馈,需将E2E输出与流利转录估计进行对比。为此提出一种新参考对齐方法,消除由流利转录错误引发的误修正。最后结合编辑置信度估计,剔除低置信度修正。在自建Linguaskill(LNG)和公开Speak & Improve(S&I)语料库上的实验表明,所提方法显著提升端到端语音语法纠错性能。

原文摘要 · Abstract (English)

Grammatical Error Correction (GEC) and feedback play a vital role in supporting second language (L2) learners, educators, and examiners. While written GEC is well-established, spoken GEC (SGEC), aiming to provide feedback based on learners' speech, poses additional challenges due to disfluencies, transcription errors, and the lack of structured input. SGEC systems typically follow a cascaded pipeline consisting of Automatic Speech Recognition (ASR), disfluency detection, and GEC, making them vulnerable to error propagation across modules. This work examines an End-to-End (E2E) framework for SGEC and feedback generation, highlighting challenges and possible solutions when developing these systems. Cascaded, partial-cascaded and E2E architectures are compared, all built on the Whisper foundation model. A challenge for E2E systems is the scarcity of GEC labeled spoken data. To address this, an automatic pseudo-labeling framework is examined, increasing the training data from 77 to over 2500 hours. To improve the accuracy of the SGEC system, additional contextual information, exploiting the ASR output, is investigated. Candidate feedback of their mistakes is an essential step to improving performance. In E2E systems the SGEC output must be compared with an estimate of the fluent transcription to obtain the feedback. To improve the precision of this feedback, a novel reference alignment process is proposed that aims to remove hypothesised edits that results from fluent transcription errors. Finally, these approaches are combined with an edit confidence estimation approach, to exclude low-confidence edits. Experiments on the in-house Linguaskill (LNG) corpora and the publicly available Speak & Improve (S&I) corpus show that the proposed approaches significantly boost E2E SGEC performance.

语音纠错端到端二语学习Whisper

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。