仅用线性变换即可实现高质量语音转换，揭示自监督特征的潜在结构。
LinearVC: Linear transformations of self-supervised features through the lens of voice conversion
- 通过旋转自监督特征实现语音转换，无需复杂模型。
- 仅100维的线性投影即可达到媲美复杂方法的转换效果。
- 适合对语音转换机制和自监督表示感兴趣的科研人员。
我们提出LinearVC，一种简单的语音转换方法，揭示了自监督表征的内在结构。首先，我们证明对自监督特征进行简单线性变换即可有效实现语音转换。接着，通过约束变换方式探究特征空间几何性质，发现仅需旋转特征即可实现高质量转换。这表明内容信息嵌入在低维子空间中，可通过线性变换映射为目标语音。为验证该假设，我们提出一种基于奇异值分解（SVD）显式分离内容与说话人信息的方法，仅需100维线性投影即获得具有竞争力的转换结果。本工作对实际语音转换及自监督语音表征的理解具有重要意义。代码与样例：https://www.kamperh.com/linearvc/。
原文摘要 · Abstract (English)
We introduce LinearVC, a simple voice conversion method that sheds light on the structure of self-supervised representations. First, we show that simple linear transformations of self-supervised features effectively convert voices. Next, we probe the geometry of the feature space by constraining the set of allowed transformations. We find that just rotating the features is sufficient for high-quality voice conversion. This suggests that content information is embedded in a low-dimensional subspace which can be linearly transformed to produce a target voice. To validate this hypothesis, we finally propose a method that explicitly factorizes content and speaker information using singular value decomposition; the resulting linear projection with a rank of just 100 gives competitive conversion results. Our work has implications for both practical voice conversion and a broader understanding of self-supervised speech representations. Samples and code: https://www.kamperh.com/linearvc/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。