开源音质评估工具,能精准判断AI生成音乐的单段质量。
MuQ-Eval: An Open-Source Per-Sample Quality Metric for AI Music Generation Evaluation
- 用冻结的MuQ-310M特征+轻量模型,实现每段音乐独立评分。
- 在31个系统上达到0.957(系统级)和0.838(片段级)的人类相关性。
- 支持少量标注数据微调,适合个性化听感评测,全开源可实时运行。
分布度量如弗雷歇音频距离无法对单个音乐片段打分,且与人类判断相关性差;现有唯一高相关性逐样本学习度量为闭源。本文提出MUQ-EVAL,基于31个文生音乐系统生成片段及专家评分的MusicEval数据集,利用冻结的MuQ-310M特征训练轻量预测头,实现开源逐样本音质评估。最简模型(冻结特征+注意力池化+两层MLP)在系统级与片段级分别取得0.957和0.838的斯皮尔曼等级相关系数(SRCC)。消融实验表明,无需额外改进,冻结基线已充分捕获质量相关信息,编码器选择是决定性因素。使用仅150个样本微调的LoRA模型即可达到可用相关性,支持基于个人标注的定制化评价器。受控退化分析显示其对信号级伪影敏感,但对音乐结构扭曲不敏感。该工具完全开源,性能优于现有开源方法,可在单张消费级显卡上实时运行。代码、模型权重与评估脚本见:https://github.com/dgtql/MuQ-Eval。
原文摘要 · Abstract (English)
Distributional metrics such as Fréchet Audio Distance cannot score individual music clips and correlate poorly with human judgments, while the only per-sample learned metric achieving high human correlation is closed-source. We introduce MUQ-EVAL, an open-source per-sample quality metric for AIgenerated music built by training lightweight prediction heads on frozen MuQ-310M features using MusicEval, a dataset of generated clips from 31 text-to-music systems with expert quality ratings. Our simplest model, frozen features with attention pooling and a two-layer MLP, achieves system-level SRCC = 0.957 and utterance-level SRCC = 0.838 with human mean opinion scores. A systematic ablation over training objectives and adaptation strategies shows that no addition meaningfully improves the frozen baseline, indicating that frozen MuQ representations already capture quality-relevant information. Encoder choice is the dominant design factor, outweighing all architectural and training decisions. LoRA-adapted models trained on as few as 150 clips already achieve usable correlation, enabling personalized quality evaluators from individual listener annotations. A controlled degradation analysis reveals selective sensitivity to signal-level artifacts but insensitivity to musical-structural distortions. Our metric, MUQ-EVAL, is fully open-source, outperforms existing open per-sample metrics, and runs in real time on a single consumer GPU. Code, model weights, and evaluation scripts are available at https://github.com/dgtql/MuQ-Eval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。