arXiv:2606.24169cs.AI2026-06

低数据时多语言初始化更优,高数据后效果趋同。

Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR

  • 用多语言或单语英语模型初始化,影响取决于目标语言数据量。
  • 100小时数据下多语言优势达4.21个百分点,2500小时后几乎消失。
  • 量化与延迟可独立优化,不依赖初始化选择。

将流式语音识别模型适配新语言时,需在多语言(ML)或仅英语(EN)编码器初始化间抉择。实验在8种欧洲语言上,对0.6亿参数的缓存感知FastConformer转换器进行系统测试,覆盖100至2500小时目标语言数据、三种流式层级及离线解码,使用多个公开测试集。结果表明:多语言初始化仅为低数据下的优势,而非延迟相关。在FLEURS数据集160毫秒延迟下,100小时时多语言优于单语4.21个百分点,2500小时时差距降至0.20个百分点;数据每翻倍,优势约减半。在三类流式层级中,100至1000小时间平均差距稳定,2500小时后接近零。4比特权重量化在560毫秒流式层级下使编码器体积缩小约3倍,平均FLEURS WER上升约0.5个百分点。结论清晰:低数据用多语言初始化,大数据下选择无关紧要,延迟与量化决策可独立处理。

原文摘要 · Abstract (English)

Adapting a streaming speech recognition model to a new language requires choosing between two plausible warm starts: a multilingual (ML) encoder or an English-only (EN) encoder. The common intuition is that the multilingual encoder should help most at low data, but it is unclear how long that advantage persists, whether tight streaming latency amplifies it, and whether it survives deployment quantization. We answer these questions with a controlled sweep of a 0.6 B-parameter cache-aware FastConformer transducer across eight European languages, up to five target-language data scales (100 h to 2500 h), three streaming tiers plus offline decoding, and up to four public test sets. The main result is that multilingual initialization is a data-limited advantage, not a latency-limited one. On FLEURS at 160 ms, the mean EN-ML word error rate (WER) gap falls from +4.21 percentage points (pp) at 100 h to +0.20 pp at 2500 h; a power-law fit summarizes this decay, with each doubling of target-language data roughly halving the remaining advantage. Across the three streaming tiers, the across-language mean EN-ML gap is approximately stable at each scale from 100 to 1000 h, and is near zero by 2500 h. Finally, 4-bit weight-only encoder quantization at the matched 560 ms streaming tier reduces the encoder footprint by about 3x, with an average FLEURS WER increase of about 0.5 pp. The resulting guideline is simple: use multilingual initialization in low-data regimes, treat the choice as effectively irrelevant at large data, and make latency and quantization decisions independently.

语音识别多语言流式处理量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。