用强化学习训练的AI实时伴奏，让人类与机器合奏更自然
ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers
- AI通过预测演奏走势提前规划，视觉提示让用户感知其意图
- 真实音乐人测试显示合奏体验愉悦且富有创意
- 适合对实时交互音乐系统感兴趣的开发者和音乐创作者
生成式人工智能已能生成高质量音乐内容，但极少关注其在实时或协作式合奏场景中的应用。此类场景需低延迟、动作预判沟通及实时适应用户输入等关键能力。为此，我们提出ReaLJam，一个基于Transformer的AI代理与人类进行现场合奏的界面与协议，该代理通过强化学习训练。系统采用前瞻机制，使AI持续预测表演走向，并以可视化方式向用户传达其计划。我们组织了用户研究，邀请经验丰富的音乐人通过ReaLJam与AI实时合奏。结果表明，ReaLJam可实现令人愉悦且音乐性丰富的合奏体验，并为未来工作提供了重要启示。
原文摘要 · Abstract (English)
Recent advances in generative artificial intelligence (AI) have created models capable of high-quality musical content generation. However, little consideration is given to how to use these models for real-time or cooperative jamming musical applications because of crucial required features: low latency, the ability to communicate planned actions, and the ability to adapt to user input in real-time. To support these needs, we introduce ReaLJam, an interface and protocol for live musical jamming sessions between a human and a Transformer-based AI agent trained with reinforcement learning. We enable real-time interactions using the concept of anticipation, where the agent continually predicts how the performance will unfold and visually conveys its plan to the user. We conduct a user study where experienced musicians jam in real-time with the agent through ReaLJam. Our results demonstrate that ReaLJam enables enjoyable and musically interesting sessions, and we uncover important takeaways for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。