arXiv:2507.20900cs.SDcs.AI2025-07被引 7

打造实时音乐生成评估平台,让用户对比文本生成音乐效果并公开偏好数据。

Music Arena: Live Evaluation for Text-to-Music

  • 用户输入文本提示,实时比较两个音乐生成模型输出
  • 收集听感数据与自然语言反馈,构建可更新的偏好榜单
  • 专为音乐生成设计路由系统与隐私保护机制,适合研究者和开发者使用

我们提出 Music Arena,一个开放的、可扩展的人类偏好评估平台,用于文本到音乐(TTM)模型的实时评价。目前,通过听觉实验获取人类偏好是 TTM 评估的黄金标准,但这类实验成本高且难以横向比较,因协议差异大。此外,人类偏好有助于对齐模型或改进自动评估指标,但当前缺乏公开、可持续的偏好数据源。Music Arena 通过提供实时评估解决上述问题:真实用户自定义文本提示,比较两个 TTM 系统的输出,并将偏好纳入排行榜。平台遵循其他 AI 领域的评估趋势,同时针对音乐特性设计了基于 LLM 的路由系统以处理不同模型的类型签名差异,并收集详细偏好数据,包括听觉记录与自然语言反馈。我们还提出滚动数据发布策略,在保障用户隐私的前提下提供可再生的偏好数据,提升平台透明度。通过标准化协议、透明数据政策和音乐专用功能,Music Arena 不仅解决了 TTM 生态中的关键挑战,也展示了如何将实时评估适配特定 AI 领域。平台地址:https://music-arena.org;偏好数据获取:https://huggingface.co/music-arena。

原文摘要 · Abstract (English)

We present Music Arena, an open platform for scalable human preference evaluation of text-to-music (TTM) models. Soliciting human preferences via listening studies is the gold standard for evaluation in TTM, but these studies are expensive to conduct and difficult to compare, as study protocols may differ across systems. Moreover, human preferences might help researchers align their TTM systems or improve automatic evaluation metrics, but an open and renewable source of preferences does not currently exist. We aim to fill these gaps by offering *live* evaluation for TTM. In Music Arena, real-world users input text prompts of their choosing and compare outputs from two TTM systems, and their preferences are used to compile a leaderboard. While Music Arena follows recent evaluation trends in other AI domains, we also design it with key features tailored to music: an LLM-based routing system to navigate the heterogeneous type signatures of TTM systems, and the collection of *detailed* preferences including listening data and natural language feedback. We also propose a rolling data release policy with user privacy guarantees, providing a renewable source of preference data and increasing platform transparency. Through its standardized evaluation protocol, transparent data access policies, and music-specific features, Music Arena not only addresses key challenges in the TTM ecosystem but also demonstrates how live evaluation can be thoughtfully adapted to unique characteristics of specific AI domains. Music Arena is available at: https://music-arena.org . Preference data is available at: https://huggingface.co/music-arena .

文本生成音乐人类偏好实时评估音乐AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。