将克赫尼图案实时转为音调,实现声音与视觉的精准互动。
ChladniSonify: A Visual-Acoustic Mapping Method for Chladni Patterns in New Media Art Creation

- 基于板振动理论构建数据集,用轻量CNN识别节点线特征。
- 分类准确率99.33%,端到端延迟低于50毫秒,频率映射无偏差。
- 适合新媒体艺术创作者,提供可复现的音画交互工具。
在新媒体艺术创作中,视觉与听觉的映射常具主观性。作为经典声学可视化载体,克赫尼图案在构建音画映射机制方面潜力巨大。然而现有工具存在仿真技术门槛高、离线计算难支持实时交互、通用音化工具映射规则不可控等痛点。为此,本文提出ChladniSonify,一种面向克赫尼图案的实时音画映射方法。基于基尔霍夫-洛夫板理论,通过数值编程构建配对数据集,并利用ANSYS有限元仿真进行校准。聚焦克赫尼图案纤细的节点线特征,采用带CBAM模块的轻量级CNN实现高精度、低延迟的模式分类。最终在Python与Max/MSP中搭建端到端系统,将识别出的图案映射为对应正弦波频率。实验结果表明,该系统具备良好可用性:分类模块在测试集上达99.33%准确率,推理延迟仅7.03毫秒;映射频率与理论值完全一致,无偏差;平均端到端延迟低于50毫秒,满足实时交互需求。本工作为克赫尼音画艺术创作提供了可复现的工程原型。
原文摘要 · Abstract (English)
In new media art creation, the mapping between vision and hearing is often subjective. As a classic carrier of sound visualization, Chladni patterns have great potential in building audio-visual mapping mechanisms. However, existing tools face pain points: high technical barriers for simulation, offline computing failing real-time interaction, and uncontrollable mapping rules in general sonification tools. To address these, this paper proposes ChladniSonify, a real-time visual-acoustic mapping method for Chladni patterns. Based on Kirchhoff-Love plate theory, we build a paired dataset via numerical programming and calibrate it using ANSYS finite element simulation. Focusing on the slender nodal lines of Chladni patterns, we adopt a lightweight CNN with CBAM to achieve high-precision, low-latency pattern classification. Finally, we build an end-to-end system in Python and Max/MSP, mapping recognized patterns to corresponding sine wave frequencies. Results show the system has excellent usability: the classification module achieves 99.33% accuracy on the test set with 7.03 ms inference latency; the mapped frequency matches the theoretical value with zero deviation; the average end-to-end latency is under 50 ms, meeting real-time interactive needs. This work provides a reproducible engineering prototype for Chladni audio-visual art creation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。