用自监督语音表示实现无需人工标注的语音对齐质量评估
Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation
- 基于自监督语音表征，通过音素聚类和词重复一致性计算新指标
- 在85种语言上验证，与人工标注结果高度相关且能区分对齐质量高低
- 适合大规模、多语言语音分析，尤其适用于低资源语种研究
强制对齐评估通常依赖人工标注的时间戳，限制了大规模和多语言分析。本文提出两种基于自监督（SSL）语音表示的无参考式对齐评估指标：音素聚类互信息（PCMI）和词声学一致性得分（WACS）。PCMI衡量对齐音素标签与从SSL语音表示中聚类得到的音素簇之间的一致性；WACS则利用动态时间规整比较词表示序列间的相似性，评估重复词实现的一致性。通过随机和系统性扰动实验，证明两者在对齐扰动下均呈一致下降趋势。我们在来自FLEURS的85种语言上分析多个对齐系统，在DoReCo的45种语言上与人工标注对齐进行对比，并在两种音系复杂的低资源语言上进行评估。结果表明，这些指标能有效区分高质量与低质量对齐，且与基于时间戳的评估指标强相关。研究表明，自监督语音表示可实现可扩展的无参考式强制对齐评估。相关指标已开源，可通过https://github.com/mahesh-ak/forced-aligner-metrics获取。
原文摘要 · Abstract (English)
Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at https://github.com/mahesh-ak/forced-aligner-metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。