用贝叶斯方法实现实时音乐拼贴,支持创作中动态调整音色与节奏。
The Concatenator: A Bayesian Approach To Real Time Concatenative Musaicing
- 以粒子滤波推断音频库窗口状态,实现实时拼贴
- 支持任意长度音频库,计算复杂度与库大小无关
- 适合现场演出和模块化合成,可实时调节音高、粒度和匹配度
我们提出「The Concatenator」,一个基于贝叶斯框架的实时音频引导拼贴系统。类似 Driedger 等人提出的「musaicing」技术,该系统从音频库中选取固定数量的音频窗,重构目标音频的谐波与打击乐特征。不同于其基于 NMF 的方法,本系统将音频库窗口索引视为隐状态,目标音频作为观测,采用粒子滤波实现实时推断。过渡模型包含可调参数以控制音块的时间连续性,观测模型允许用户优先考虑窗口切换速度以匹配目标。由于系统计算复杂度与音频库大小无关,可扩展至数小时长的音频数据集。在系统内,作曲者可实时调整音粒长度、对齐精度与音高偏移,快速迭代创作想法。我们通过大量定量测试参数影响,并进行艺术性定性评估。结果表明,实时能力为音乐表达与控制开辟新途径,适用于现场表演与模块化合成集成,标志着拼贴合成技术的重要突破。
原文摘要 · Abstract (English)
We present ``The Concatenator,'' a real time system for audio-guided concatenative synthesis. Similarly to Driedger et al.'s ``musaicing'' (or ``audio mosaicing'') technique, we concatenate a set number of windows within a corpus of audio to re-create the harmonic and percussive aspects of a target audio stream. Unlike Driedger's NMF-based technique, however, we instead use an explicitly Bayesian point of view, where corpus window indices are hidden states and the target audio stream is an observation. We use a particle filter to infer the best hidden corpus states in real-time. Our transition model includes a tunable parameter to control the time-continuity of corpus grains, and our observation model allows users to prioritize how quickly windows change to match the target. Because the computational complexity of the system is independent of the corpus size, our system scales to corpora that are hours long, which is an important feature in the age of vast audio data collections. Within The Concatenator module itself, composers can vary grain length, fit to target, and pitch shift in real time while reacting to the sounds they hear, enabling them to rapidly iterate ideas. To conclude our work, we evaluate our system with extensive quantitative tests of the effects of parameters, as well as a qualitative evaluation with artistic insights. Based on the quality of the results, we believe the real-time capability unlocks new avenues for musical expression and control, suitable for live performance and modular synthesis integration, which furthermore represents an essential breakthrough in concatenative synthesis technology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。