通过可学习的全局标记实现语音的无监督可控编辑
Exploring Token-Space Manipulation in Latent Audio Tokenizers
- 在音频特征序列末尾添加可学习的固定标记，形成全局信息聚合瓶颈
- 仅保留这些标记进行量化与解码，在低比特率下仍保持良好重建质量
- 无需监督即可通过交换标记位置实现说话人身份、背景噪声等属性修改
神经音频编解码器为语音生成与编辑提供紧凑的离散表示。然而，大多数编解码器将标记组织为帧级序列，难以对全局变化因素进行研究或干预。本文提出潜空间音频标记器（LATTE），在音频特征序列末尾添加一组固定的可学习潜变量标记，并仅对这些标记进行量化与解码。该设计生成一个紧凑且非时序对齐的瓶颈，使每个标记能聚合整个话语的全局信息。我们证明，该标记器在低比特率语音编码场景中仍保持竞争力的重建质量，同时支持简单的潜空间干预。特别地，我们发现交换不同话语间的特定潜变量标记位置，可改变说话人身份和背景噪声等全局属性，并在语音转换与降噪任务中进行了评估。结果表明，紧凑的潜空间音频标记器可在无需监督的情况下，支持任务特异性的可控音频编辑。
原文摘要 · Abstract (English)
Neural audio codecs provide compact discrete representations for speech generation and manipulation. However, most codecs organize tokens as frame-level sequences, making it difficult to study or intervene on global factors of variation. In this work, we propose the Latent Audio Tokenizer for Token-space Editing (LATTE) that appends a fixed set of learnable latent tokens to the audio feature sequence and retains only these tokens for quantization and decoding. This design produces a compact, non-temporally aligned bottleneck in which each token can aggregate global information across the full utterance. We show that the resulting tokenizer preserves competitive reconstruction quality in low-bitrate speech coding settings while enabling simple token-space interventions. In particular, we find that swapping selected latent token positions between utterances can modify global attributes, such as speaker identity and background noise, and we evaluate these interventions on voice conversion and denoising tasks. Our results suggest that compact latent audio tokenizers can support controllable audio manipulation without supervision in task-specific editing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。