用文本去噪方式适配大模型语音识别，不破坏语音文本对齐。
Text-only adaptation in LLM-based ASR through text denoising
- 将文本适配看作去噪任务，训练语言模型恢复干净文本。
- 在两个数据集上相对提升最高达22.1%，优于现有方法。
- 无需修改结构或增加参数，轻量高效，适合快速迁移。
使用纯文本数据将基于大语言模型（LLM）的自动语音识别（ASR）系统适配到新领域是一个重要但未被充分探索的挑战。在目标领域文本上对LLM进行标准微调常会破坏语音与文本模态间通过投影器学习的关键对齐关系，导致性能下降。本文提出一种新颖的纯文本适配方法，将该过程建模为文本去噪任务：训练LLM从带噪声输入中恢复干净转录。该方法在有效适应目标领域的同时，保持了跨模态对齐。方案轻量，无需架构改动或引入额外参数。在两个数据集上的大量实验表明，相对提升最高可达22.1%，优于近期最先进的纯文本适配方法。
原文摘要 · Abstract (English)
Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet underexplored challenge. Standard fine-tuning of the LLM on the target domain text often disrupts the critical alignment between the speech and text modality learned by the projector, degrading performance. We introduce a novel text-only adaptation method that frames this process as a text denoising task. Our approach trains the LLM to recover clean transcripts from noisy inputs. This process effectively adapts the model to a target domain while preserving cross-modal alignment. Our solution is lightweight, requiring no architectural changes or additional parameters. Extensive evaluation on two datasets demonstrates up to 22.1% relative improvement, outperforming recent state-of-the-art text-only adaptation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。