无需微调即可提升语音模型对生僻词的识别与翻译能力
Rare Word Recognition and Translation Without Fine-Tuning via Task Vector in Speech Models
- 通过任务向量差值定义生僻词能力，实现参数无损增补
- 在多个领域上达到或超越微调模型效果，通用性能提升约5 BLEU
- 支持灵活组合，适合需要快速部署新词汇的场景
生僻词仍是语音转文本系统的关键瓶颈。直接微调虽能提升目标词识别，但代价高、易导致灾难性遗忘且难以扩展。为此，我们提出一种基于任务向量的无训练范式，用于生僻词识别与翻译。通过将任务向量定义为参数差异，并引入词级任务向量运算，该方法可灵活组合生僻词能力，显著提升可扩展性与复用性。在多个领域的大量实验表明，该方法在目标词上的表现达到或超过微调模型，通用性能提升约5 BLEU，同时有效缓解灾难性遗忘。
原文摘要 · Abstract (English)
Rare words remain a critical bottleneck for speech-to-text systems. While direct fine-tuning improves recognition of target words, it often incurs high cost, catastrophic forgetting, and limited scalability. To address these challenges, we propose a training-free paradigm based on task vectors for rare word recognition and translation. By defining task vectors as parameter differences and introducing word-level task vector arithmetic, our approach enables flexible composition of rare-word capabilities, greatly enhancing scalability and reusability. Extensive experiments across multiple domains show that the proposed method matches or surpasses fine-tuned models on target words, improves general performance by about 5 BLEU, and mitigates catastrophic forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。