通过扩展上下文窗口提升语音识别对实体的准确识别与格式化能力
Mind the Gap: Entity-Preserved Context-Aware ASR Structured Transcriptions
- 训练时引入5秒重叠的40秒语义窗口，增强长距离上下文理解
- 实体跨分块时整体归入右侧片段，确保格式正确性，显著降低错误率
- 使用带标签的增强数据训练，让模型同时学会识别和按类型格式化实体
自动语音识别（ASR）系统如Whisper虽具备高转录准确率，但在命名实体和数值数据处理上表现不佳，尤其在需特定格式的法律、金融和医疗等关键领域。本文提出一种新型训练方法：在训练中对30秒语音片段两侧各添加5秒重叠，形成40秒的有效语义窗口，聚焦中央30秒的预测结果。针对跨越分块边界的实体，将其完整归入右邻块以保证格式正确。同时，利用含实体标签的增强数据集，使模型学习实体识别与类型化格式。在Spoken Wikipedia数据集上的评估显示，该方法在命名实体识别（NER）和实体格式化等语义任务上均有提升，证明上下文感知训练对长文本转录和复杂实体识别的有效性。
原文摘要 · Abstract (English)
Automatic Speech Recognition (ASR) systems, such as Whisper, achieve high transcription accuracy but struggle with named entities and numerical data, especially when proper formatting is required. These issues increase word error rate (WER) and impair semantic understanding in critical domains like legal, financial, and medical applications. We propose a novel training approach that extends the semantic context of ASR models by adding overlapping context windows during training. By sliding 5-second overlaps on both sides of 30-second chunks, we create a 40-second "effective semantic window," improving entity recognition and formatting while focusing predictions on the central 30 seconds. To address entities spanning chunk boundaries, we reassign such entities entirely to the right-hand chunk, ensuring proper formatting. Additionally, enriched training data with embedded entity labels enables the model to learn both recognition and type-specific formatting. Evaluated on the Spoken Wikipedia dataset, our method improves performance across semantic tasks, including named entity recognition (NER) and entity formatting. These results highlight the effectiveness of context-aware training in addressing ASR limitations for long-form transcription and complex entity recognition tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。