arXiv:2511.14939cs.SDcs.LG2025-11

用音频模型检测新冠,发现数据偏差会虚高效果,真实表现很差。

Fine-tuning Pre-trained Audio Models for COVID-19 Detection: A Technical Report

  • 用年龄性别分层防数据偏见,让评估更真实
  • 跨数据集测试时模型性能暴跌,最高AUC仅0.68
  • 小样本加严格控制,难训练出可靠语音新冠检测模型

本技术报告研究预训练音频模型在新冠检测任务上的表现,使用Coswara和COUGHVID两个基准数据集。对Audio-MAE及三种PANN架构(CNN6、CNN10、CNN14)进行微调,并评估了模型在同数据集与跨数据集上的泛化能力。通过按年龄和性别严格分层,防止模型利用人口统计特征与新冠状态之间的虚假关联。同数据集结果中,Audio-MAE在Coswara上表现最佳(AUC 0.82,F1-score 0.76),但所有模型在Coughvid上表现有限(AUC 0.58–0.63)。跨数据集评估显示严重泛化失败(AUC 0.43–0.68),其中Audio-MAE的F1-score降至0.00–0.08。实验表明,尽管人口统计平衡降低了模型表现,却能消除因人口泄漏导致的性能虚高,提供更真实的评估。此外,平衡后数据量仅为1,219–2,160样本,远低于深度学习模型所需规模,难以支撑有效训练。这些发现揭示了构建可泛化语音新冠检测系统的根本挑战,强调在临床评估中需采用严格的分层控制。

原文摘要 · Abstract (English)

This technical report investigates the performance of pre-trained audio models on COVID-19 detection tasks using established benchmark datasets. We fine-tuned Audio-MAE and three PANN architectures (CNN6, CNN10, CNN14) on the Coswara and COUGHVID datasets, evaluating both intra-dataset and cross-dataset generalization. We implemented a strict demographic stratification by age and gender to prevent models from exploiting spurious correlations between demographic characteristics and COVID-19 status. Intra-dataset results showed moderate performance, with Audio-MAE achieving the strongest result on Coswara (0.82 AUC, 0.76 F1-score), while all models demonstrated limited performance on Coughvid (AUC 0.58-0.63). Cross-dataset evaluation revealed severe generalization failure across all models (AUC 0.43-0.68), with Audio-MAE showing strong performance degradation (F1-score 0.00-0.08). Our experiments demonstrate that demographic balancing, while reducing apparent model performance, provides more realistic assessment of COVID-19 detection capabilities by eliminating demographic leakage - a confounding factor that inflate performance metrics. Additionally, the limited dataset sizes after balancing (1,219-2,160 samples) proved insufficient for deep learning models that typically require substantially larger training sets. These findings highlight fundamental challenges in developing generalizable audio-based COVID-19 detection systems and underscore the importance of rigorous demographic controls for clinically robust model evaluation.

新冠检测音频模型数据偏差泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。