将音乐去噪过程变成可实时演奏的乐器,支持快速响应与多参数控制。
DEMON: Diffusion Engine for Musical Orchestrated Noise

- 每槽独立调度去噪时序,实现滑块移动不中断处理队列
- 单卡实测生成速度达12.3次/秒(60秒音乐),支持低延迟演奏
- 适合音乐人、实时音频创作者,打造可交互的生成式音乐工具
我们提出DEMON,一种实时扩散引擎,使去噪过程可作为现场演奏的音乐仪器:控制界面既广泛(每帧可调节多个输出参数)又灵敏(每个控制响应速度可达去噪循环允许的极限)。基于ACE-Step 1.5和StreamDiffusion的环形缓冲架构,并采用TensorRT加速,在单张消费级显卡(RTX 5090)上可实现每秒12.3次解码完成(60秒音乐),或在生产环深4时达11.3次/秒。此时去噪参数可作为现场表演控制,但环形缓冲仅在出水速率下传播请求变更,最低为S步。我们贡献四种机制:(1) 每槽异构去噪调度:各环槽拥有独立时间步序列,移动去噪滑块无需清空在途队列;(2) 共享可变每步状态,使任一参数在每求解步立即生效,绕过环缓出水延迟;(3) 每帧源混合:对标准SDE重噪声步骤施加采样时刻控制,提供帧级变换强度轴,补充标量去噪调度;(4) 基于感受野分析的窗口化VAE解码,带来8.0倍解码加速。这四类机制将流式扩散参数按触发与收敛延迟划分为四类传播类别。
原文摘要 · Abstract (English)
We present DEMON, a real-time diffusion engine that makes the denoising process playable as a live musical instrument: a control surface both broad (many parameters shaped per-frame across the output) and responsive (each control taking effect as fast as its place in the denoising loop allows). Built on ACE-Step 1.5 and StreamDiffusion's ring-buffer architecture with TensorRT acceleration, it sustains up to 12.3 decoder completions per second for 60-second music on a single consumer GPU (RTX 5090), or 11.3 generations per second at our production ring-depth of 4. At these rates denoising parameters become viable as live performance controls, but the ring buffer propagates per-request changes only at its drain rate, a floor of S denoising steps. We contribute four mechanisms. (1) Per-slot heterogeneous denoise scheduling: each ring-buffer slot owns its timestep schedule, so a moving denoise slider is tracked without wiping the in-flight queue, where the upstream global-schedule design must rebuild and discard it. (2) Shared mutable per-step state, giving any parameter consulted at every solver step next-tick effect, bypassing ring-buffer drain. (3) Per-frame source blending: a sampling-time control on the standard SDE re-noise step, giving a framewise transformation-strength axis that complements scalar denoise scheduling. (4) Windowed VAE decode exploiting receptive-field analysis for an 8.0x decode speedup. Together these separate streaming-diffusion parameters into four propagation classes, by onset and convergence latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。