用稀疏自编码器揭示音频模型内部表示，提升可解释性
Sparse Autoencoders Make Audio Foundation Models more Explainable
- 用稀疏自编码器分析预训练音频模型的隐藏表示
- 保留原始表征与类别标签信息，揭示自监督学习机制
- 增强声乐特征解耦，适合研究模型可解释性的学者
音频预训练模型广泛应用于语音处理、声音事件检测和音乐信息检索等任务。然而，这些模型所学的表示难以理解，现有分析方法主要依赖对隐藏表示的线性探测。本文探索使用稀疏自编码器（SAEs）分析预训练模型的隐藏表示，聚焦于歌唱技巧分类任务。我们首先证明，SAEs能同时保留原始表示和类别标签信息，使模型内部结构可提供对自监督学习系统的洞察。此外，SAEs显著提升了声乐属性的解耦能力，证实其是识别表示中潜在因素的有效工具。
原文摘要 · Abstract (English)
Audio pretrained models are widely employed to solve various tasks in speech processing, sound event detection, or music information retrieval. However, the representations learned by these models are unclear, and their analysis mainly restricts to linear probing of the hidden representations. In this work, we explore the use of Sparse Autoencoders (SAEs) to analyze the hidden representations of pretrained models, focusing on a case study in singing technique classification. We first demonstrate that SAEs retain both information about the original representations and class labels, enabling their internal structure to provide insights into self-supervised learning systems. Furthermore, we show that SAEs enhance the disentanglement of vocal attributes, establishing them as an effective tool for identifying the underlying factors encoded in the representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。