用书法笔触实时生成音乐，让书写动作直接控制旋律与和声。
Calliphony: A Calligraphy-Driven Interface for Real-Time Generative Music Performance
- 通过笔触传感器捕捉书写动作，转化为音乐控制信号。
- 实时生成多轨MIDI，支持音符密度与和声层动态调节。
- 适合跨媒体表演者与实验性音乐创作者使用。
尽管音乐生成模型近期备受关注，但如何将其有效融入现场表演仍需深入探索。本文提出Calliphony，一种基于书法动作的实时生成音乐表演接口。我们构建了一个低延迟系统，通过可附加传感器捕捉毛笔运动，并将其映射为实时符号音乐生成的控制信号。系统利用生成模型在演出环境中生成多轨MIDI，笔触衍生的控制信号用于约束事件时机并激活额外音乐层次。生成的旋律随后通过实时和声与附加声部扩展，并最终通过DAW实现现场呈现。Calliphony贡献：（1）一个以表演为导向的原型，将书法动作作为外部控制层，用于实时符号音乐生成模型，实现音符密度、音高约束与伴奏层激活的控制；（2）一种跨模态表演场景，将书法从视觉艺术拓展至视听融合、人工智能辅助的创作环境。
原文摘要 · Abstract (English)
While music generative models have recently gained significant attention, how they can be effectively integrated into live music performances still requires further exploration. This paper presents Calliphony, a calligraphy-driven interface for real-time generative music performance. Specifically, we build a low-latency pipeline that captures brush motion with an attachable sensor and maps it to control signals for real-time symbolic music generation. Using a generative model, the system produces multi-track MIDI in performance settings, while brush-derived control signals constrain event timing and activate additional musical layers. The generated melody is then extended with real-time harmony and additional voices, and finally rendered through a DAW for live staging. Calliphony contributes: (1) a performance-oriented prototype that uses calligraphic motion as an external control layer for a real-time symbolic music generation model, controlling note density, pitch constraints, and accompaniment-layer activation; and (2) a cross-modal performance scenario that extends calligraphy beyond a primarily visual practice into an audiovisual, AI-assisted setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。