为语音邻域嵌入提供理论框架,实现跨模态语音文字匹配与方言聚类。
A Theoretical Framework for Acoustic Neighbor Embeddings
- 基于音素相似性定义距离,构建概率化嵌入解释框架。
- 在50万词汇量下,语音文字最近邻搜索准确率媲美有限状态转换器。
- 可预测唤醒词混淆,适用于语音识别与人机交互优化场景。
本文提出一种声学邻域嵌入的理论框架,用于解释可变长度语音或文本在固定维度嵌入空间中的表征。基于音素相似性的定量定义,提出嵌入间距离的概率解释,使嵌入的理解与应用更具理论依据。理论与实证表明,簇内各向同性近似成立,可将距离简化为欧氏距离。四个实验验证了该框架:语音与文本嵌入的最近邻搜索在50万词汇量下达到与有限状态转换器(FSTs)相同的孤立词分类准确率;嵌入距离在未登录词恢复任务中仅比音素编辑距离低0.5个百分点;英语方言聚类生成的层次结构与人类听觉实验结果一致。框架还可用于预测设备唤醒词的预期混淆。所有源代码与预训练模型均已公开。
原文摘要 · Abstract (English)
This paper provides a theoretical framework for interpreting acoustic neighbor embeddings, which are representations of the phonetic content of variable-width audio or text in a fixed-dimensional embedding space. A probabilistic interpretation of the distances between embeddings is proposed, based on a general quantitative definition of phonetic similarity between words. This provides us a framework for understanding and applying the embeddings in a principled manner. Theoretical and empirical evidence to support an approximation of uniform cluster-wise isotropy are shown, which allows us to reduce the distances to simple Euclidean distances. Four experiments that validate the framework and demonstrate how it can be applied to diverse problems are described. Nearest-neighbor search between audio and text embeddings can give isolated word classification accuracy that is identical to that of finite state transducers (FSTs) for vocabularies as large as 500k. Embedding distances give accuracy with 0.5% point difference compared to phone edit distances in out-of-vocabulary word recovery, as well as producing clustering hierarchies identical to those derived from human listening experiments in English dialect clustering. The theoretical framework also allows us to use the embeddings to predict the expected confusion of device wake-up words. All source code and pretrained models are provided.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。