用大模型实现可对话的语音质量评估，支持自然语言提问和解释。
SpeechQualityLLM: LLM-Based Multimodal Assessment of Speech Quality
- 结合音频编码器与语言模型，通过问答方式评估语音质量。
- 双端模式下语音质量评分误差为0.41，相关性达0.86。
- 支持自然语言交互，可模拟不同听者偏好，减少人工测试成本。
客观语音质量评估在电话、VoIP和流媒体系统中至关重要，需大规模监控和优化大量受损音频。传统指标如PESQ和POLQA虽能近似人类主观评分（MOS），但要求严格控制条件且依赖昂贵的听感测试；基于学习的模型如NISQA可从波形或频谱中回归MOS及多个感知维度，相关性高，但缺乏交互能力且无法生成文本解释。本文提出SpeechQualityLLM，一种多模态语音质量问答系统，将音频编码器与语言模型结合，基于NISQA语料库训练，覆盖整体MOS及噪声、音色失真、中断、响度四个感知维度，在单端（仅损）与双端（损+参考）设置下生成问答对。系统不直接回归分数，而是监督生成文本答案，再从中解析数值并用标准回归与排序指标评估。在留出的NISQA片段上，双端模型达到MOS平均绝对误差0.41，皮尔逊相关系数0.86，维度任务表现也具竞争力。除量化提升外，系统提供灵活的自然语言接口，语言模型作为音频质量专家：用户可查询任意退化特征，提示模型模拟不同听者类型，生成多样而合理的判断，从而降低对大规模众包测试的依赖及其成本。
原文摘要 · Abstract (English)
Objective speech quality assessment is central to telephony, VoIP, and streaming systems, where large volumes of degraded audio must be monitored and optimized at scale. Classical metrics such as PESQ and POLQA approximate human mean opinion scores (MOS) but require carefully controlled conditions and expensive listening tests, while learning-based models such as NISQA regress MOS and multiple perceptual dimensions from waveforms or spectrograms, achieving high correlation with subjective ratings yet remaining rigid: they do not support interactive, natural-language queries and do not natively provide textual rationales. In this work, we introduce SpeechQualityLLM, a multimodal speech quality question-answering (QA) system that couples an audio encoder with a language model and is trained on the NISQA corpus using template-based question-answer pairs covering overall MOS and four perceptual dimensions (noisiness, coloration, discontinuity, and loudness) in both single-ended (degraded only) and double-ended (degraded plus clean reference) setups. Instead of directly regressing scores, our system is supervised to generate textual answers from which numeric predictions are parsed and evaluated with standard regression and ranking metrics; on held-out NISQA clips, the double-ended model attains a MOS mean absolute error (MAE) of 0.41 with Pearson correlation of 0.86, with competitive performance on dimension-wise tasks. Beyond these quantitative gains, it offers a flexible natural-language interface in which the language model acts as an audio quality expert: practitioners can query arbitrary aspects of degradations, prompt the model to emulate different listener profiles to capture human variability and produce diverse but plausible judgments rather than a single deterministic score, and thereby reduce reliance on large-scale crowdsourced tests and their monetary cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。