构建平台让音乐模型与人的听觉判断对齐，首次用于印度卡纳提克音乐
DAIRHuM: A Platform for Directly Aligning AI Representations with Human Musical Judgments applied to Carnatic Music
- 搭建可交互平台，让用户标注音乐相似性并评估模型表现
- 发现模型在卡纳提克音乐节奏和谐判断上与人有显著差异
- 推动数据稀缺的南印度音乐领域实现人机对齐研究
量化并对齐音乐人工智能模型表示与人类行为是音乐信息检索（MIR）领域的重要挑战。本文提出一个名为DAIRHuM的平台，旨在探索人工智能音乐模型表示与人类音乐判断之间的直接对齐。该平台使音乐家和实验者能够标注音乐录音数据集中的相似性，并通过定量评分和可视化图表检验预训练模型与这些标注的一致性。将DAIRHuM应用于分析NSynth模型表示与卡纳提克四重奏中两位打击乐手的节奏二重奏之间的对齐情况，这是注释数据稀缺且对齐评估复杂的音乐类型。结果揭示了模型在节奏和谐判断上与人类感知存在显著差异，凸显了卡纳提克音乐中节奏感知与音乐相似性判断的独特性。本工作是首次支持用户探索卡纳提克音乐中人机对齐的研究尝试，推动印度音乐领域的MIR发展，并应对数据稀缺与文化特异性问题。平台开发增强了对代表性不足音乐类型音乐人工智能工具的可及性。
原文摘要 · Abstract (English)
Quantifying and aligning music AI model representations with human behavior is an important challenge in the field of MIR. This paper presents a platform for exploring the Direct alignment between AI music model Representations and Human Musical judgments (DAIRHuM). It is designed to enable musicians and experimentalists to label similarities in a dataset of music recordings, and examine a pre-trained model's alignment with their labels using quantitative scores and visual plots. DAIRHuM is applied to analyze alignment between NSynth representations, and a rhythmic duet between two percussionists in a Carnatic quartet ensemble, an example of a genre where annotated data is scarce and assessing alignment is non-trivial. The results demonstrate significant findings on model alignment with human judgments of rhythmic harmony, while highlighting key differences in rhythm perception and music similarity judgments specific to Carnatic music. This work is among the first efforts to enable users to explore human-AI model alignment in Carnatic music and advance MIR research in Indian music while dealing with data scarcity and cultural specificity. The development of this platform provides greater accessibility to music AI tools for under-represented genres.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。