用声音定位技术提升语音大模型对专有名词的识别准确率
CLAR: CIF-Localized Alignment for Retrieval-Augmented Speech LLM-Based Contextual ASR
- 通过连续积分-放电机制实现声文逐词对齐，无需时间戳
- 在真实语音中精准定位短实体，降低误识别率
- 适合需要高精度命名实体识别的语音识别场景
基于语音大模型的自动语音识别在处理专有名词和长尾词汇时，常因语言模型先验偏好而表现不佳。检索增强型偏置方法虽有帮助，但其效果依赖于在弱监督下对完整语句中关键词的精准定位。本文提出CLAR，一种双编码器声文检索模型，采用连续积分-放电（CIF）机制学习无时间戳的单调词级对齐。结合长度感知的局部匹配策略，CLAR能有效锚定短实体的声学线索，缓解表征稀释与注意力漂移问题。模型训练采用多粒度目标，融合全局与局部片段对比损失及CIF数量约束。推理时，将排名靠前的关键词作为上下文提示注入语音大模型，提升识别效果。实验表明，CLAR显著提升了关键词检索性能，并在字符错误率（CER）和单词错误率（B-WER）上优于强基线模型。
原文摘要 · Abstract (English)
Speech LLM-based ASR often struggles with named entities and long-tail words due to strong internal language-model priors. Retrieval-augmented biasing can help, but its effectiveness depends on accurate hotword localization in full-utterance speech under weak supervision. We propose CLAR, a dual-encoder speech-text retriever that uses Continuous Integrate-and-Fire (CIF) to learn monotonic token-level alignments without timestamps. With length-aware localized matching, CLAR anchors short-entity acoustic cues and reduces representation dilution and attention drift. The retriever is trained with a multi-granularity objective combining global and local segment-level contrastive losses and a CIF quantity constraint. At inference, top-ranked hotwords are injected as contextual prompts for the Speech LLM, improving recognition without shallow fusion. Experiments show that CLAR significantly improves hotword retrieval and reduces both CER and B-WER against strong contextual ASR baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。