用大模型生成音频可视化编程代码,发现元编程更准确、更复杂。
Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages
- 用元编程和JSON直接生成节点两种方式测试代码质量
- 元编程生成的代码语义更正确,且随机与循环提升复杂度
- 适合媒体艺术创作新手或低代码工具开发者
基于节点的编程语言在媒体艺术编程领域日益流行,其设计旨在让无编程经验的用户也能轻松实现创意输出。利用大语言模型(LLM)进行代码生成可进一步降低创作门槛,但如何高效生成这类语言的代码仍不明确。本文在多个抽象层级上评估了LLM在音频编程任务中的表现,探索了通过元编程代码表示(即使用另一种高级文本语言编写该语言)以及直接通过JSON生成节点的方式。我们在两个音频可视化编程语言的基准数据集上进行了评估,衡量生成代码的正确性和复杂性。结果表明,在代码语法正确且可运行的前提下,元编程生成的代码在语义上更准确;同时,通过提示引入随机性和循环结构,可生成更复杂的代码。
原文摘要 · Abstract (English)
Node-based programming languages are increasingly popular in media arts coding domains. These languages are designed to be accessible to users with limited coding experience, allowing them to achieve creative output without an extensive programming background. Using LLM-based code generation to further lower the barrier to creative output is an exciting opportunity. However, the best strategy for code generation for visual node-based programming languages is still an open question. In particular, such languages have multiple levels of representation in text, each of which may be used for code generation. In this work, we explore the performance of LLM code generation in audio programming tasks in visual programming languages at multiple levels of representation. We explore code generation through metaprogramming code representations for these languages (i.e., coding the language using a different high-level text-based programming language), as well as through direct node generation with JSON. We evaluate code generated in this way for two visual languages for audio programming on a benchmark set of coding problems. We measure both correctness and complexity of the generated code. We find that metaprogramming results in more semantically correct generated code, given that the code is well-formed (i.e., is syntactically correct and runs). We also find that prompting for richer metaprogramming using randomness and loops led to more complex code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。