AI与艺术家共创作沉浸式声音装置，持续生成新声景。
'Studies for': A Human-AI Co-Creative Sound Artwork Using a Real-time Multi-channel Sound Generation Model
- 用8通道实时生成模型，结合艺术家反馈迭代
- 基于200小时作品训练，生成新颖未听过的音效
- 为艺术家留下可延续的数字声学遗产
本文通过与声音艺术家Evala合作，开发了名为Studies for的生成式声音装置。该装置采用SpecMaskGIT这一轻量级高质量声音生成模型，在为期三个月的展览中实时生成八通道音频，打造沉浸式听觉体验。作品以‘新形式档案’为核心理念，旨在保留艺术家风格的同时，持续生成新声音元素，突破传统艺术作品存档的局限。模型基于超过200小时的Evala过往声音作品数据集进行训练。研究强调了人机共创中的三个关键：艺术家反馈整合、源自艺术家历史作品的数据集构建，以及确保输出中包含意外且新颖的内容。本工作提出了一种有效的人机协同创作框架，拓展了声音艺术创作与存档的可能性，使艺术家的作品能在其物理存在之外持续延展。演示页面：https://sony.github.io/studies-for/
原文摘要 · Abstract (English)
This paper explores the integration of AI technologies into the artistic workflow through the creation of Studies for, a generative sound installation developed in collaboration with sound artist Evala (https://www.ntticc.or.jp/en/archive/works/studies-for/). The installation employs SpecMaskGIT, a lightweight yet high-quality sound generation AI model, to generate and playback eight-channel sound in real-time, creating an immersive auditory experience over the course of a three-month exhibition. The work is grounded in the concept of a "new form of archive," which aims to preserve the artistic style of an artist while expanding beyond artists' past artworks by continued generation of new sound elements. This speculative approach to archival preservation is facilitated by training the AI model on a dataset consisting of over 200 hours of Evala's past sound artworks. By addressing key requirements in the co-creation of art using AI, this study highlights the value of the following aspects: (1) the necessity of integrating artist feedback, (2) datasets derived from an artist's past works, and (3) ensuring the inclusion of unexpected, novel outputs. In Studies for, the model was designed to reflect the artist's artistic identity while generating new, previously unheard sounds, making it a fitting realization of the concept of "a new form of archive." We propose a Human-AI co-creation framework for effectively incorporating sound generation AI models into the sound art creation process and suggest new possibilities for creating and archiving sound art that extend an artist's work beyond their physical existence. Demo page: https://sony.github.io/studies-for/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。