让语音大模型高效识别新名词,不靠提示词,也不易出错。
Beyond Prompting: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integration (LOGIC)
- 在解码层直接注入上下文信息,不依赖提示词。
- 实体错误率降低9%,误报率仅上升0.30%。
- 适合需要实时识别新术语的语音交互场景。
新实体的快速涌现——由文化变迁、趋势演化及个性化用户数据驱动——对现有语音大语言模型(Speech LLMs)构成重大挑战。尽管这些模型在通用对话任务中表现优异,但其静态训练知识限制了对联系人姓名、播放列表或专业术语等领域的识别能力。现有方法主要依赖提示词,但存在可扩展性差的问题:随着实体列表增长,提示词会遭遇上下文窗口限制、推理延迟增加以及‘中间丢失’现象。另一种方案生成式纠错(GEC)通过后处理重写转录文本,却常出现‘过度纠正’,引入从未被说出的实体幻觉。本文提出LOGIC(Logit-Space Integration for Contextual Biasing),一种在解码层直接运行的高效鲁棒框架。与提示词不同,LOGIC将上下文注入与输入处理解耦,实现相对于提示长度的恒定时间复杂度。在11个语种环境下使用Phi-4-MM模型的大量实验表明,LOGIC平均实现9%的相对实体词错误率降低,且误报率仅增加0.30%。
原文摘要 · Abstract (English)
The rapid emergence of new entities -- driven by cultural shifts, evolving trends, and personalized user data -- poses a significant challenge for existing Speech Large Language Models (Speech LLMs). While these models excel at general conversational tasks, their static training knowledge limits their ability to recognize domain-specific terms such as contact names, playlists, or technical jargon. Existing solutions primarily rely on prompting, which suffers from poor scalability: as the entity list grows, prompting encounters context window limitations, increased inference latency, and the "lost-in-the-middle" phenomenon. An alternative approach, Generative Error Correction (GEC), attempts to rewrite transcripts via post-processing but frequently suffers from "over-correction", introducing hallucinations of entities that were never spoken. In this work, we introduce LOGIC (Logit-Space Integration for Contextual Biasing), an efficient and robust framework that operates directly in the decoding layer. Unlike prompting, LOGIC decouples context injection from input processing, ensuring constant-time complexity relative to prompt length. Extensive experiments using the Phi-4-MM model across 11 multilingual locales demonstrate that LOGIC achieves an average 9% relative reduction in Entity WER with a negligible 0.30% increase in False Alarm Rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。