跨语言语音转发音运动预测性能下降更严重，提出新评估基准
Beyond Speaker Independence: Evaluating Cross-Lingual Acoustic-to-Articulatory Inversion Across Finnish and Russian
- 构建芬兰语-俄语双语发音运动数据集FROST-EMA，打破英语主导
- 跨语言迁移导致相关性下降0.10~0.20，比跨性别仅降0.05~0.10更严重
- 首次定义跨性别与跨语言迁移的统一评估协议，适合语音建模研究者
在说话人属性与跨语言条件变化导致领域偏移时，语音到发音运动逆向映射（AAI）仍具挑战性。本文系统评估此类偏移，在芬兰语-俄语双语发音运动（FROST-EMA）数据集上建立基线。该数据集缓解了现有资源对英语的依赖及说话人多样性不足的问题。我们对比了（i）发音目标（原始EMA坐标与声道变量）、（ii）声学前端（MFCC与自监督学习特征）、（iii）逆向映射后端（BiLSTM与轻量级注意力序列模型）。进一步定义了跨性别迁移（同语言内）与跨语言迁移（同性别内）的评估协议。结果表明，跨性别不匹配导致皮尔逊相关性下降约0.05至0.10，而跨语言不匹配引起更大降幅（约0.10至0.20）。
原文摘要 · Abstract (English)
Acoustic-to-articulatory inversion (AAI) remains challenging under domain shifts where changes in speaker attributes and cross-language conditions often degrade performance. We conduct a systematic evaluation under such shifts and establish baseline benchmarks on FROST-EMA, a Finnish-Russian bilingual EMA corpus. FROST-EMA addresses the English bias and limited speaker diversity of existing resources. We benchmark (i) articulatory targets (raw EMA coordinates vs tract variables), (ii) acoustic front-ends (MFCC vs SSL features), and (iii) inversion back-ends (BiLSTM vs a lightweight attention-based sequence model). We further define evaluation protocols for cross-gender transfer (within language) and cross-language transfer (within gender). The results indicate that cross-gender mismatch introduces moderate Pearson correlation declines (approximately 0.05 to 0.10) relative to the in-domain baseline, whereas cross-language mismatch causes larger drops (approximately 0.10 to 0.20).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。