用嵌入排名无监督评估自监督语音模型，省去标注数据和调参。
Towards Automatic Assessment of Self-Supervised Speech Models using Rank
- 引入嵌入排名作为无需下游标签的评估指标。
- 排名与下游任务表现相关，但无法稳定预测最佳层。
- 适合监控训练过程，降低评估资源消耗。
本研究探索使用嵌入排名作为通用语音编码器在自监督学习（SSL）中无监督评估的指标。传统评估需大量标注数据和下游任务微调，成本高昂。受视觉领域启发，该工作考察嵌入排名在语音领域的适用性，考虑信号的时间特性。结果表明，嵌入排名在不同下游任务及域内/域外场景下，与编码器各层的下游性能具有相关性。然而，排名无法可靠预测特定任务的最佳层，低排名层有时表现优于高排名层。尽管存在局限，研究仍表明嵌入排名可作为监控SSL语音模型训练进程的有效工具，提供比传统方法更少资源依赖的替代方案。
原文摘要 · Abstract (English)
This study explores using embedding rank as an unsupervised evaluation metric for general-purpose speech encoders trained via self-supervised learning (SSL). Traditionally, assessing the performance of these encoders is resource-intensive and requires labeled data from the downstream tasks. Inspired by the vision domain, where embedding rank has shown promise for evaluating image encoders without tuning on labeled downstream data, this work examines its applicability in the speech domain, considering the temporal nature of the signals. The findings indicate rank correlates with downstream performance within encoder layers across various downstream tasks and for in- and out-of-domain scenarios. However, rank does not reliably predict the best-performing layer for specific downstream tasks, as lower-ranked layers can outperform higher-ranked ones. Despite this limitation, the results suggest that embedding rank can be a valuable tool for monitoring training progress in SSL speech models, offering a less resource-demanding alternative to traditional evaluation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。