用高斯混合模型锚定语音表示，避免自监督学习中的表征坍缩。
Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
- 用一次软聚类生成固定目标，替代反复迭代聚类。
- 在5万小时语音上，语音识别错误率降低4.54个百分点。
- 适合追求稳定训练和高效聚类的语音表征研究者。
联合嵌入预测架构（JEPA）为自监督语音表示学习提供了有前景的方案，但缺乏显式约束时易出现表征坍缩。本文提出GMM-Anchored JEPA，先对对数梅尔频谱图拟合一次高斯混合模型，并将其冻结的软后验作为训练全程的辅助目标。采用衰减式监督策略，使GMM正则化在早期主导训练，随后逐渐让位于JEPA目标。与HuBERT和WavLM不同，本方法仅需一次软聚类，无需迭代重聚类。在约5万小时语音数据上，相比计算量相当的WavLM基线，该方法在语音识别（WER从28.68%降至33.22%）、情感识别（准确率从67.76%升至65.46%）和槽位填充（F1从64.7%升至59.1%）任务中均有提升。聚类分析显示，GMM锚定表示的熵高达98%，远超WavLM风格的31%，表明聚类利用更均匀。代码已开源。
原文摘要 · Abstract (English)
Joint Embedding Predictive Architectures (JEPA) offer a promising approach to self-supervised speech representation learning, but suffer from representation collapse without explicit grounding. We propose GMM-Anchored JEPA, which fits a Gaussian Mixture Model once on log-mel spectrograms and uses its frozen soft posteriors as auxiliary targets throughout training. A decaying supervision schedule allows GMM regularization to dominate early training before gradually yielding to the JEPA objective. Unlike HuBERT and WavLM, which require iterative re-clustering, our approach clusters input features once with soft rather than hard assignments. On ~50k hours of speech, GMM anchoring improves ASR (28.68% vs. 33.22% WER), emotion recognition (67.76% vs. 65.46%), and slot filling (64.7% vs. 59.1% F1) compared to a WavLM-style baseline with matched compute. Cluster analysis shows GMM-anchored representations achieve up to 98% entropy compared to 31% for WavLM-style, indicating substantially more uniform cluster utilization. Code is made available at https://github.com/gioannides/clustering-anchored-jepa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。