用上下文增强的分词判别法，提升语音搜索纠错效果
Contextualized Token Discrimination for Speech Search Query Correction
- 用BERT生成分词上下文表示，再融合语义信息
- 通过对比原始与上下文表示，精准识别并纠正错误分词
- 在新构建的语音转录错误数据集上表现更优，适合语音搜索研究
查询拼写纠错是现代搜索引擎的重要功能，能有效帮助用户清晰表达意图。随着语音搜索因自动语音识别（ASR）系统日益普及，本文提出一种名为上下文分词判别（Contextualized Token Discrimination, CTD）的新方法，用于高效进行语音查询纠错。CTD首先利用BERT生成分词级别的上下文表示，再通过组合层增强语义信息，最后根据聚合后的分词表示，通过比较原始分词表示与上下文表示，修正错误分词。大量实验表明，该方法在所有指标上均优于现有方法。此外，本文还构建了一个包含错误ASR转录的新基准数据集，为语音查询纠错提供全面评估支持。
原文摘要 · Abstract (English)
Query spelling correction is an important function of modern search engines since it effectively helps users express their intentions clearly. With the growing popularity of speech search driven by Automated Speech Recognition (ASR) systems, this paper introduces a novel method named Contextualized Token Discrimination (CTD) to conduct effective speech query correction. In CTD, we first employ BERT to generate token-level contextualized representations and then construct a composition layer to enhance semantic information. Finally, we produce the correct query according to the aggregated token representation, correcting the incorrect tokens by comparing the original token representations and the contextualized representations. Extensive experiments demonstrate the superior performance of our proposed method across all metrics, and we further present a new benchmark dataset with erroneous ASR transcriptions to offer comprehensive evaluations for audio query correction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。