用主成分扩散模型实现符号化鼓点到音频的精准对齐生成
Seconds-Aligned PCA-DAC Latent Diffusion for Symbolic-to-Audio Drum Rendering
- 基于主成分分析压缩潜空间，以连续坐标预测音频特征
- 在1733个四拍片段上，频谱与瞬态指标优于传统回归方法
- 适合需要高时序精度的音乐生成任务，尤其擅长短步长扩散
符号化控制的鼓点生成需在保留明确事件时间与动态的同时，合成声学合理的波形。我们提出Sec2Drum-DAC，一种用于符号到音频鼓点渲染的条件潜空间扩散模型。该模型在编码帧位置上以物理时间采样事件特征，并预测冻结的DAC和码本嵌入的标准化主成分坐标，而非直接生成波形样本。在评估的DAC配置中，72个主成分在给定的SVD阈值下捕获了训练帧的和潜空间子空间，形成紧凑的连续去噪目标，并具备确定性的重构路径至1024维的DAC潜空间，再进行波形解码。在1,733个独立的四拍窗口上，主成分扩散在配对的频谱与瞬态指标上优于确定性主成分回归和符号渲染基线，而直接回归在对相位敏感的波形L1指标上仍更优。辅助的RVQ交叉熵在梅尔误差、起音通量余弦及波形L1上提升了短步长扩散效果，最优权衡出现在6-25个去噪步之间，具体取决于评价指标。
原文摘要 · Abstract (English)
Symbolic-control drum generation requires preserving explicit event timing and dynamics while synthesizing acoustically plausible waveforms. We present Sec2Drum-DAC, a conditional latent-diffusion model for symbolic-to-audio drum rendering. The model conditions on event features sampled in physical time at codec-frame locations and predicts standardized principal-component coordinates of frozen DAC summed-codebook embeddings rather than waveform samples. In the evaluated DAC configuration, 72 principal components capture the observed training-frame summed-latent subspace under the stated SVD threshold, yielding a compact continuous denoising target with a deterministic reconstruction path to the 1024-dimensional DAC latent space before waveform decoding. Across 1,733 held-out four-beat windows, PCA diffusion improves paired spectral and transient metrics over deterministic PCA regression and a symbolic rendering baseline, while direct regression remains stronger on phase-sensitive waveform L1. Auxiliary RVQ cross-entropy improves short-step diffusion on mel error, onset-flux cosine, and waveform L1, with the most favorable trade-offs occurring at 6-25 denoising steps depending on the metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。