arXiv:2606.21018cs.SDcs.AI2026-06

实时人机共演音乐生成,低延迟互动突破传统限制。

LK Jam: System Architecture and Implementation of a Real-Time Human-AI Interactive Music Generation System using Role-Aware GRU

论文配图:LK Jam: System Architecture and Implementation of a Real-Time Human-AI Interactive Music Generation System using Role-Aware GRU
图 1 · 摘自论文原文
  • 用轻量GRU与事件流建模,单步推理实现精准轮换与微时序
  • 端到端延迟恒定在O(1),零内存分配保障音频线程不卡顿
  • 三阶段训练达专家级交互水平,适合现场音乐合作场景

随着具身智能的发展,现场音乐互动亟需突破离线、单向生成的局限,实现低延迟、动态协同的“虚拟默契”。本文提出LK_Jam,一个基于轻量门控循环单元(GRU)与高性能音频宿主架构的实时双向人机音乐生成系统。算法层摒弃计算开销大的固定时间网格,构建融合时间偏移、连续和声嵌入及角色感知编码的多维稀疏事件流,支持单步推理准确捕捉轮次逻辑与微时序。工程层采用C++与JUCE框架构建无锁多线程通信桥梁,集成专为实时音频设计的RTNeural推理引擎,通过编译期网络拓扑固化与零分配机制,将自回归解码的端到端开销严格控制在O(1)复杂度,从根本上规避了DAW插件环境中的音频线程丢帧风险。此外,设计三阶段渐进式训练策略,实现从基础和声到专家级交互的跃升。初步观测与架构分析表明,该系统在保证音乐连贯性与角色扮演的同时,成功应对极端实时性挑战,为下一代实时AI合作者提供了高鲁棒、可部署的技术范式。

原文摘要 · Abstract (English)

As artificial intelligence advances into the era of Embodied AI, live musical interaction urgently needs to break free from the limitations of offline, unidirectional generation, achieving a "virtual synergy" capable of low-latency, dynamic interplay. To address this, this technical report presents LK_Jam, a real-time, bidirectional human-computer interactive music generation system based on a lightweight Gated Recurrent Unit (GRU) and a high-performance audio host architecture. In the algorithmic representation layer, this system abandons the computationally expensive fixed time-grid. Instead, it constructs a multi-dimensional sparse event stream integrating time-shifts, continuous harmonic embeddings, and role-aware encoding, enabling the model to accurately capture turn-taking logic and micro-timing in a single-step inference. In the engineering implementation layer, this paper builds a strict multithreaded lock-free communication bridge using C++ and the JUCE framework, incorporating the RTNeural inference engine designed specifically for real-time audio. By utilizing compile-time network topology solidification and a zero-allocation (allocation-free) mechanism, the end-to-end overhead of autoregressive decoding is strictly locked at \(O(1)\) complexity, structurally mitigating the risk of audio thread dropouts in DAW plugin environments. Furthermore, this study designs a three-stage progressive training strategy, achieving a leap from basic chord harmonization to expert-level interaction. Preliminary observations and architectural analysis demonstrate that while ensuring musical coherence and interactive role-play, the proposed system successfully challenges extreme real-time engineering constraints, offering a highly robust and deployable technical paradigm for next-generation AI co-performers in live music.

音乐生成实时系统人机交互GRU

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。