用微调让多语模型更懂日语语音识别，效果超过专用模型。
Efficient Adaptation of Multilingual Models for Japanese ASR
- 用LoRA和端到端训练微调Whisper-Tiny模型
- 日语字符错误率从32.7降至14.7，优于Whisper-Base
- 适合资源少、书写复杂的语言快速适配
本研究探索了对多语种语音识别模型Whisper-Tiny进行微调，以提升其在日语上的表现。尽管多语模型具备通用性，但在特定语言上精度不足；而专用模型如ReazonSpeech虽性能优异，却缺乏可扩展性。通过使用日语专属数据集，结合低秩适应（LoRA）与端到端训练，我们对Whisper-Tiny进行了微调。结果表明，采用LoRA后字符错误率（CER）从32.7降至20.8，端到端微调进一步降至14.7，优于Whisper-Base的20.2。然而，领域专有术语仍存在识别挑战，凸显专用数据集的重要性。研究证明，微调多语模型可在保持灵活性的同时实现强语言特异性表现，为资源受限及复杂书写系统语言（如日语）提供可扩展的语音识别解决方案。
原文摘要 · Abstract (English)
This study explores fine-tuning multilingual ASR (Automatic Speech Recognition) models, specifically OpenAI's Whisper-Tiny, to improve performance in Japanese. While multilingual models like Whisper offer versatility, they often lack precision in specific languages. Conversely, monolingual models like ReazonSpeech excel in language-specific tasks but are less adaptable. Using Japanese-specific datasets and Low-Rank Adaptation (LoRA) along with end-to-end (E2E) training, we fine-tuned Whisper-Tiny to bridge this gap. Our results show that fine-tuning reduced Whisper-Tiny's Character Error Rate (CER) from 32.7 to 20.8 with LoRA and to 14.7 with end-to-end fine-tuning, surpassing Whisper-Base's CER of 20.2. However, challenges with domain-specific terms remain, highlighting the need for specialized datasets. These findings demonstrate that fine-tuning multilingual models can achieve strong language-specific performance while retaining their flexibility. This approach provides a scalable solution for improving ASR in resource-constrained environments and languages with complex writing systems like Japanese.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。