fine-tuning wav2vec 2.0 可隐式实现语音规范化，无需显式预处理。
Normalization through Fine-tuning: Understanding Wav2vec 2.0 Embeddings for Phonetic Analysis
- 通过微调wav2vec 2.0，自动抑制无关任务信息实现语音特征归一化。
- 多任务微调模型能保留所有任务信息且不影响分类性能。
- 揭示了语音模型与人类语音感知中归一化的潜在机制。
语音规范化在语音识别与分析中至关重要，确保原始音频提取特征的可比性。然而，在当前预训练大模型微调范式下，语音规范化不再被视为必要步骤，而是由模型隐式完成。本研究深入分析了变压器模型（特别是wav2vec 2.0）中的归一化过程。通过对多个任务微调的模型嵌入进行综合分析，结果表明：微调wav2vec 2.0可通过选择性抑制任务无关信息，有效实现语音规范化。我们发现，多任务微调模型能同时保留各任务信息且不降低性能；抑制无关信息并非实现有效分类的必要条件。这些发现为语音模型中灵活实现语音规范化提供了新视角，并揭示了其在人类语音感知中的实现机制。
原文摘要 · Abstract (English)
Phonetic normalization plays a crucial role in speech recognition and analysis, ensuring the comparability of features derived from raw audio data. However, in the current paradigm of fine-tuning pre-trained large transformer models, phonetic normalization is not deemed a necessary step; instead, it is implicitly executed within the models. This study investigates the normalization process within transformer models, especially wav2vec 2.0. Through a comprehensive analysis of embeddings from models fine-tuned for various tasks, our results demonstrate that fine-tuning wav2vec 2.0 effectively achieves phonetic normalization by selectively suppressing task-irrelevant information. We found that models fine-tuned for multiple tasks retain information for both tasks without compromising performance, and that suppressing task-irrelevant information is not necessary for effective classification. These findings provide new insights into how phonetic normalization can be flexibly achieved in speech models and how it is realized in human speech perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。