arXiv:2510.08392eess.AScs.SD2025-10被引 7

提出轻量级流式零样本语音转换模型,单步生成高质量语音。

MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows

  • 采用分块自回归去噪的扩散变换器,兼顾效率与流畅性。
  • 仅需1次采样即完成转换,音质和说话人相似度显著提升。
  • 参数量少、推理快,适合实时语音应用,开源可复现。

零样本语音转换(VC)旨在将源说话人的音色迁移到任意未见目标说话人,同时保持语言内容不变。日益增长的应用需求要求模型具备流式推理能力。现有方法多依赖自回归(AR)或非自回归(NAR)框架,前者参数量大,后者难以泛化到未见说话人。本文提出轻量级、流式的零样本语音转换模型 MeanVC。该模型采用分块自回归去噪的扩散变换器,融合 AR 与 NAR 优势以实现高效流式处理。通过引入均值流(mean flows),在训练中回归平均速度场,使模型能在单次采样步骤内直接从起点映射至终点,实现零样本转换。此外,引入扩散对抗后训练以缓解过平滑问题,进一步提升语音质量。实验表明,MeanVC 显著优于现有流式零样本 VC 系统,在转换质量、效率和参数量方面均有明显优势。音频演示与代码已公开于 https://aslp-lab.github.io/MeanVC。

原文摘要 · Abstract (English)

Zero-shot voice conversion (VC) aims to transfer timbre from a source speaker to any unseen target speaker while preserving linguistic content. Growing application scenarios demand models with streaming inference capabilities. This has created a pressing need for models that are simultaneously fast, lightweight, and high-fidelity. However, existing streaming methods typically rely on either autoregressive (AR) or non-autoregressive (NAR) frameworks, which either require large parameter sizes to achieve strong performance or struggle to generalize to unseen speakers. In this study, we propose MeanVC, a lightweight and streaming zero-shot VC approach. MeanVC introduces a diffusion transformer with a chunk-wise autoregressive denoising strategy, combining the strengths of both AR and NAR paradigms for efficient streaming processing. By introducing mean flows, MeanVC regresses the average velocity field during training, enabling zero-shot VC with superior speech quality and speaker similarity in a single sampling step by directly mapping from the start to the endpoint of the flow trajectory. Additionally, we incorporate diffusion adversarial post-training to mitigate over-smoothing and further enhance speech quality. Experimental results demonstrate that MeanVC significantly outperforms existing zero-shot streaming VC systems, achieving superior conversion quality with higher efficiency and significantly fewer parameters. Audio demos and code are publicly available at https://aslp-lab.github.io/MeanVC.

语音转换扩散模型轻量化流式处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。