让音乐图文模型学会拒绝回答,提升判断可靠性。
Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models

- 通过打乱选项顺序构建伪集成,不需重训练
- 准确率从55.7%提升至59.2%,错误识别更准
- 适合资源有限的紧凑型音乐多模态模型使用
音乐音频-语言模型几乎仅通过多项选择题的准确率评估,迫使模型必须作答,导致偶然猜测与真实理解难以区分。缺乏识别不确定性的能力,使模型无法在不知情时放弃回答。传统方法依赖多个独立训练的模型集成,成本过高。本文提出从单个预训练模型构建伪集成:通过不影响正确答案的方式扰动输入(如打乱选项顺序、音频噪声、标签交换),并平均多个预测分布。该方法支持完整的集成不确定性度量(期望熵、熵的期望及其差值互信息),而不仅是单一分布熵。在TinyMU模型上测试,对4种选项顺序平均后,准确率由55.7%升至59.2%,误差保留曲线下面积从0.293降至0.261。整个过程仅增加少量前向传播,无需重新训练,使小型模型实现实用的拒答机制。
原文摘要 · Abstract (English)
Music audio-language models are evaluated almost entirely by accuracy on multiple-choice questions. This protocol forces the model to commit to an option, so a lucky guess looks the same as real musical understanding. What is missing is a way to tell when the model does not know the answer, so that it can abstain instead of guessing. The usual solution, an ensemble of independently trained models, is far too expensive here, which leaves the entropy of a single predictive distribution as the only available confidence signal. We instead build pseudo-ensembles from one pretrained model by perturbing its input in ways that cannot change the correct answer, then averaging the resulting distributions over the options. Our main construction simply shuffles the order in which the candidate answers are presented; we also study ensembles built from corrupted audio and from swapped option labels. A pseudo-ensemble gives several predictive distributions per question, so it supports the full family of ensemble-based uncertainty measures (entropy of the expected distribution, expected entropy, and their difference, the mutual information) rather than entropy alone. Evaluating TinyMU on MuChoMusic, we find that averaging over four option orderings raises accuracy from 55.7% to 59.2%, and that the resulting uncertainty measures rank the model's errors better than the single-pass entropy baseline, reducing the area under the error retention curve from 0.293 to 0.261. All of this costs a few extra forward passes and no retraining, which makes abstention practical for compact music audio-language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。