用扩散模型噪声空间估算音乐意外性，效果优于传统方法。
Estimating Musical Surprisal from Audio in Autoregressive Diffusion Model Noise Spaces
- 在扩散模型噪声空间中计算信息含量以估计音乐意外性。
- 在单音旋律和多轨音频边界检测任务中表现超越GIVT。
- 不同噪声水平对应不同音频粒度的意外性，可调优提升效果。
近期，生成式无限词汇变压器（GIVT）预测的信息内容（IC）被用于建模音频中的音乐期待与意外性。本文研究了基于自回归扩散模型（ADMs）计算的IC在该任务中的有效性。实证表明，采用两种不同扩散常微分方程（ODE）的模型，在负对数似然指标上优于GIVT，能更优地描述数据。通过两项任务评估：(1) 单音旋律的音高意外性捕捉；(2) 多轨音频段落边界的检测。在两项任务中，扩散模型的性能均达到或超过GIVT。我们假设，扩散过程不同噪声水平下估计的意外性对应于不同音频粒度下的音乐特征。验证该假设后发现，适当噪声水平下，音乐意外性任务表现显著提升。代码已开源至github.com/SonyCSLParis/audioic。
原文摘要 · Abstract (English)
Recently, the information content (IC) of predictions from a Generative Infinite-Vocabulary Transformer (GIVT) has been used to model musical expectancy and surprisal in audio. We investigate the effectiveness of such modelling using IC calculated with autoregressive diffusion models (ADMs). We empirically show that IC estimates of models based on two different diffusion ordinary differential equations (ODEs) describe diverse data better, in terms of negative log-likelihood, than a GIVT. We evaluate diffusion model IC's effectiveness in capturing surprisal aspects by examining two tasks: (1) capturing monophonic pitch surprisal, and (2) detecting segment boundaries in multi-track audio. In both tasks, the diffusion models match or exceed the performance of a GIVT. We hypothesize that the surprisal estimated at different diffusion process noise levels corresponds to the surprisal of music and audio features present at different audio granularities. Testing our hypothesis, we find that, for appropriate noise levels, the studied musical surprisal tasks' results improve. Code is provided on github.com/SonyCSLParis/audioic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。