用心跳控制音乐实时变形，打造身心共振的表演体验。
Coupling the Heart to Musical Machines
- 通过心率变化实时调整音频时间尺度，实现声音动态重塑。
- 蓝牙无线传输下，心率与声音形成闭环反馈，响应延迟低。
- 适合音乐表演、沉浸式艺术创作及生物反馈研究者使用。
生物反馈正日益成为人机交互的新范式。尽管呼吸信号在新型音乐接口（NIME）中应用广泛，但心脏信号却未受足够重视。事实上，心脏与呼吸一样，是音乐感知的核心——它影响我们对时间的体感，进而间接塑造音乐体验。受此启发，本文提出一种基于光电容积脉搏波描记法（PPG）的NIME控制器，利用心率作为一维控制参数，在蓝牙无线人机界面下实时改变声音特性。通过时间缩放技术对输入声卡的音频缓冲区进行‘扭曲’处理，并将变换后的音频回放给佩戴PPG传感器的听众，形成假设中的感知性生物反馈环路：声音变化影响心率，心率变化被PPG捕捉，再反向作用于声音。文中探讨了该反馈环如何提升一维控制器的控制力与表现多样性，以及通过生物反馈调控声音时空尺度对现场演出氛围的拓展潜力，并简要讨论生成潜在空间作为扩展一维PPG控制空间的可能性。
原文摘要 · Abstract (English)
Biofeedback is being used more recently as a general control paradigm for human-computer interfaces (HCIs). While biofeedback especially from breath has seen increasing uptake as a controller for novel musical interfaces, new interfaces for musical expression (NIMEs), the community has not given as much attention to the heart. The heart is just as intimate a part of music as breath and it is argued that the heart determines our perception of time and so indirectly our perception of music. Inspired by this I demonstrate a photoplethysmogram (PPG)-based NIME controller using heart rate as a 1D control parameter to transform the qualities of sounds in real-time over a Bluetooth wireless HCI. I apply time scaling to "warp" audio buffers inbound to the sound card, and play these transformed audio buffers back to the listener wearing the PPG sensor, creating a hypothetical perceptual biofeedback loop: changes in sound change heart rate to change PPG measurements to change sound. I discuss how a sound-heart-PPG biofeedback loop possibly affords greater control and/or variety of movements with a 1D controller, how controlling the space and/or time scale of sound playback with biofeedback makes for possibilities in performance ambience, and I briefly discuss generative latent spaces as a possible way to extend a 1D PPG control space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。