首次分析多语言配音中语调跨语言相似性，发现特定语言间存在固有语调关联。
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
- 基于英德、英西、英法配音数据，细粒度对比源语言与目标语言的语调、能量和时序特征
- 发现英德、英法对中语调结构存在显著跨语言相关性，英西对则较弱
- 为情感化语音翻译系统设计提供实证依据，适合语音合成与跨语言研究者参考
语调在语音翻译中至关重要，传递语气、情感和意图等词汇之外的信息。尽管近年来在表达性语音到语音翻译（S2ST）方面取得进展，但不同语言间语调模式的相似性与差异性仍不明确。理解这些跨语言的异同对于有效整合语调至表达性S2ST系统至关重要。本文首次利用英德、英西、英法三组语言对的多语言配音数据，开展细粒度跨语言语调分析，比较源语言与目标语言在基频、能量和时间特征上的相似性，并探究语言类型与对齐因素的影响。结果揭示了某些语言之间语调结构存在固有跨语言相关性。研究为语调在跨语言间的可迁移性提供了重要洞见，为未来表达性语音到语音翻译系统的设计提供了实证指导。
原文摘要 · Abstract (English)
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expressive speech-to-speech translation (S2ST), little is known about how prosodic patterns are similar/different across languages. Understanding these cross-lingual similarities and differences is crucial for effectively incorporating prosody into expressive S2ST systems. In this work, we present the first fine-grained cross-lingual analysis of prosody using multilingual dubbing data across English-German, English-Spanish, and English-French language pairs. We analyze the similarity of pitch, energy, and temporal feature patterns between source and target speech and investigate the linguistic and alignment-related factors affecting this similarity. Our analysis reveals inherent cross-lingual correlations in prosodic structure between certain languages. The findings provide important insights into the transferability of prosody across languages and offer empirical guidance for future expressive speech-to-speech translation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。