用正则化微调模型，10秒搞定阿拉伯语语音带符号转写。
Thaka at KSAA-2026 Task 2: Regularized Fine-Tuning for Arabic Speech Diacritization
- 用CATT+冻结Whisper的多模态模型，字符级处理语音转写
- 23.26%词错误率，全榜第一，仅用2327个训练样本
- 高权重衰减+焦点损失+蒙特卡洛丢弃，提升小样本泛化能力
我们介绍在KSAA-2026共享任务第2项中获胜的系统，该任务要求从语音音频和无符号转写文本中生成完整的阿拉伯语带符号文本。训练数据仅有2,327个样本，且禁止使用外部数据。系统采用基于预训练CATT文本编码器与冻结Whisper语音编码器的字符级多模态模型进行微调。核心方法为训练正则化：包括R-Drop一致性正则化、通过Optuna优化的高权重衰减超参数设置以及焦点损失。推理阶段，通过对四个模型检查点进行200次蒙特卡洛丢弃的随机前向传播，并在软标签层取平均。最终系统在主排行榜指标（包含词尾变体及无符号位置）上达到23.26%的词错误率（WER），位列所有参赛者首位。
原文摘要 · Abstract (English)
We describe the winning system for Task 2 of the KSAA-2026 Shared Task on Arabic Speech Dictation with Automatic Diacritization. The task requires producing fully diacritized Arabic text from speech audio and undiacritized transcripts, with only 2,327 training samples available and no external data permitted. Our system fine-tunes CATT-Whisper, a character-level multimodal model combining a pretrained CATT text encoder with a frozen Whisper speech encoder. The key to our approach is training regularization: R-Drop consistency regularization, Optuna-optimized hyperparameters with high weight decay, and Focal Loss. At inference, we average 200 stochastic forward passes across four model checkpoints using Monte Carlo Dropout at the softmax probability level. The system achieves 23.26% WER on the primary leaderboard metric (with case endings, including no-diacritic positions), placing 1st among all participants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。