用自监督模型激活映射，仅需少量标注实现语音分段与识别
Phone Segmentation and Recognition through Phonological Activation Mapping
- 基于自监督语音模型，将帧级表示映射为音位特征激活向量
- 仅需不到1分钟语音标注，即可在多个数据集上达到优秀性能
- 无需梯度下降的轻量头设计，适合低资源语音任务
语音分段与识别是密切相关但常被分开处理的任务。我们提出基于自监督语音模型（S3M）的音位激活映射（SPAM），将每个模型帧映射为包括清浊、鼻音等音位特征的激活向量。在此基础上，设计两个无需梯度下降的轻量预测头：识别头与分段头。该方法仅需不到一分钟的音素标注，且在训练中可泛化到未见音素。在多个不同数据集上均取得优异的分段与识别表现。
原文摘要 · Abstract (English)
Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce two simple but effective lightweight, gradient-descent-free prediction heads: a recognition head and a segmentation head. Our method requires less than a minute of phonetic transcriptions, and generalizes to unseen phones during training. Across a diverse range of datasets, our approach attains strong segmentation and recognition performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。