研究语音转写错误如何影响下游语言理解任务，提出可配置评估框架。
Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks
- 构建可调节噪声的评估框架，模拟不同严重程度和类型的转写错误。
- 发现模型对噪声有一定容忍度，且不同错误类型影响差异显著。
- 适用于语音理解系统优化，尤其适合关注转写质量的研究者。
随着录音人类语音的日益普及，语音语言理解（SLU）对于高效处理至关重要。为处理语音，通常使用自动语音识别技术进行转写，这一过程会引入错误，并传递至下游自然语言处理任务，如对话摘要。尽管已知转写噪声会影响下游任务，但缺乏系统性分析不同噪声强度和类型影响的方法。本文提出一种可配置的评估框架，用于在多样噪声环境下评估任务模型，并检验转写清洗技术的效果。该框架有助于揭示任务模型的行为特征，从而支持有效SLU解决方案的开发。我们在三个SLU任务和四个任务模型上验证了框架的有效性，发现模型可在一定程度上容忍噪声，且对不同类型的转写错误反应各异。
原文摘要 · Abstract (English)
With the increasing prevalence of recorded human speech, spoken language understanding (SLU) is essential for its efficient processing. In order to process the speech, it is commonly transcribed using automatic speech recognition technology. This speech-to-text transition introduces errors into the transcripts, which subsequently propagate to downstream NLP tasks, such as dialogue summarization. While it is known that transcript noise affects downstream tasks, a systematic approach to analyzing its effects across different noise severities and types has not been addressed. We propose a configurable framework for assessing task models in diverse noisy settings, and for examining the impact of transcript-cleaning techniques. The framework facilitates the investigation of task model behavior, which can in turn support the development of effective SLU solutions. We exemplify the utility of our framework on three SLU tasks and four task models, offering insights regarding the effect of transcript noise on tasks in general and models in particular. For instance, we find that task models can tolerate a certain level of noise, and are affected differently by the types of errors in the transcript.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。