无需训练，用预训练编码器实现可控音频变音
Neural Morphing: Sequence-Optimized Token-Level Morphing in Neural Audio Codecs
- 从用户指定音色库中选取残差向量量化令牌，替换原音频
- 通过分层编码组策略与连续性约束搜索，保持节奏不变
- 可实时部署于音频插件，支持动态调色板大小调整
神经音频编解码器最初用于高保真压缩，但其潜在的令牌表示和表达性解码器也为可控音频变换提供了强大基础。本文提出 Neural Morphing，一种无需训练的令牌域音频效果：从用户音色库中选择残差向量量化（RVQ）令牌颗粒，并通过预训练编解码器解码编辑后的音频流。方法结合了将粗、中、细编码组分离的 RVQ 组迁移策略，以及限制连续性的序列匹配器，将独立贪婪选择替换为有界束搜索。目标输出为可控混合音：源音频保留节奏结构，而音色库贡献音色色彩与残差细节。研究聚焦于可部署 VST3/AU 系统的实现与实时行为，包括分块渲染、音色库大小缩放及后端健康检查。
原文摘要 · Abstract (English)
Neural audio codecs were originally developed for high-fidelity compression; however, their latent token representations and expressive decoders also constitute a powerful substrate for controllable audio transformation. This work introduces Neural Morphing, a training-free token-domain audio effect that selects residual-vector-quantized (RVQ) token grains from a user palette and decodes the edited stream through a pretrained codec. The method combines an RVQ-group transfer policy that separates coarse, middle, and fine codebook groups with a continuity-constrained sequence matcher that replaces independent greedy selection with bounded beam search. The intended output is a controlled hybrid: the source preserves rhythmic organization while the palette contributes timbral color and residual detail. We focus on the implementation and realtime behavior of a deployable VST3/AU system, including chunked rendering, palette-size scaling, and backend health checks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。