用更少比较次数精准估算音频质量，提升主观评测效率。
Optimal Pairwise Comparison Procedures for Subjective Evaluation
- 通过成对比较替代评分，降低评估误差和疲劳
- 新方法在少量比较下快速收敛到真实排序
- 适合大规模音频算法评估，尤其需高效测试场景
音频信号处理算法常通过主观听觉测试进行评估，参与者在单维数值尺度上为受损信号打分。但评估者间量表校准不一致会带来偏差。成对比较提供更直观的替代方案，可减少测量误差并降低受试者疲劳。然而，由于比较数量随数据规模呈二次增长，完整成对比较对大规模数据集不可行。本文比较不同成对比较方法，寻找以最少比较逼近真实质量分数的最优策略。提出一种新型采样方法，并在模拟数据集上与现有先进方法对比。结果表明，贝叶斯采样在已有方法中表现最稳健，而所提方法在收敛速度上显著更快，且评分准确性相当。
原文摘要 · Abstract (English)
Audio signal processing algorithms are frequently assessed through subjective listening tests in which participants directly score degraded signals on a unidimensional numerical scale. However, this approach is susceptible to inconsistencies in scale calibration between assessors. Pairwise comparisons between degraded signals offer a more intuitive alternative, eliciting the relative scores of candidate signals with lower measurement error and reduced participant fatigue. Yet, due to the quadratic growth of the number of necessary comparisons, a complete set of pairwise comparisons becomes unfeasible for large datasets. This paper compares pairwise comparison procedures to identify the most efficient methods for approximating true quality scores with minimal comparisons. A novel sampling procedure is proposed and benchmarked against state-of-the-art methods on simulated datasets. Bayesian sampling produces the most robust score estimates among previously established methods, while the proposed procedure consistently converges fastest on the underlying ranking with comparable score accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。