用AI将卡德的视觉乐谱转化为可演奏的音乐。
Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
- 用ChatGPT解析视觉符号，生成文本提示。
- 通过MusicLDM生成音乐，使用外扩技术拼接段落。
- 为实验音乐创作提供新表演视角，适合创意作曲者。
本文提出一种新方法，利用AI诠释科内留斯·卡德的《手册》（Treatise）图形乐谱，实现音乐创作与即兴演奏。借助OpenAI的ChatGPT解析《手册》中的抽象视觉元素，将其转化为描述性文本提示，再输入至MusicLDM——一个预训练的潜在扩散音乐生成模型。我们引入“外扩”（outpainting）技术，通过重叠生成的音乐片段，实现无缝且连贯的旋律衔接。该方法展示了以AI转化视觉刺激为声音的新视角，拓展了当代/实验音乐创作的边界。相关音乐作品可在 https://bit.ly/TreatiseAI 查看。
原文摘要 · Abstract (English)
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI's ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。