arXiv:2504.04721eess.AS2025-04

提出随机产品量化方法,显著减少语音表示离散化时的信息损失。

Bridging the Gap between Continuous and Informative Discrete Representations by Random Product Quantization

  • 通过分块独立量化并随机采样维度,提升离散表示多样性。
  • 在LibriSpeech上相对降低21.8%的词错误率,ML-SUPERB上降低24.1%字符错误率。
  • 适合追求高效且高精度语音表示的模型部署场景。

自监督学习在语音处理中已成核心,但其高维表示需离散化以提升效率。现有方法存在显著信息损失,导致性能远低于连续表示。本文提出两种基于量化的离散化方法:产品量化(PQ)与随机产品量化(RPQ)。PQ将特征空间划分为多个子空间,分别独立量化,融合各子空间信息,缓解单簇量化损失。RPQ通过多次随机采样固定比例特征维度构造子向量,更好捕捉数据分布变异性。理论分析表明,RPQ降低了子量化器间的相关系数ρ(0 ≤ ρ ≤ 1),其量化误差下界为ρ与单个K均值量化的ε-kms乘积。在结合LibriSpeech与ML-SUPERB的数据集上实验显示,PQ与RPQ优于标准K均值离散化,在LibriSpeech上分别实现21.8%和20.0%的词错误率相对降低,在ML-SUPERB上分别实现24.1%和19.6%的字符错误率降低。其性能可媲美甚至超越连续表示。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) has become a core technique in speech processing, but the high dimensionality of its representations makes discretization essential for improving efficiency. However, existing discretization methods still suffer from significant information loss, resulting in a notable performance gap compared to continuous representations. To overcome these limitations, we propose two quantization-based discretization methods: Product Quantization (PQ) and Random Product Quantization (RPQ). PQ partitions the original feature space into multiple subspaces and independently quantizes each sub-vector, producing a fused set of discrete units that retain diverse information from different subspaces, thereby mitigating the loss associated with single-cluster quantization. RPQ further enhances representation diversity by randomly sampling a fixed proportion of feature dimensions multiple times to construct sub-vectors, thereby better capturing the variability in the data distribution. Theoretical analysis shows that RPQ reduces the correlation coefficient rho (where 0 <= rho <= 1) between sub-quantizers. Its quantization error is lower-bounded by the product of rho and epsilon-kms, where epsilon-kms denotes the quantization error of a single K-means quantizer. Experimental results on a combined dataset built from LibriSpeech and ML-SUPERB show that PQ and RPQ outperform standard K-means discretization, achieving relative improvements of 21.8 percent and 20.0 percent in WER on LibriSpeech, and 24.1 percent and 19.6 percent in CER on ML-SUPERB, respectively. Moreover, their performance is competitive with, and in some cases even surpasses, that of continuous SSL representations.

语音处理离散化量化解析自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。