用Whisper自动分析多语老年人访谈中的说话人角色与语言使用，辅助评估语言能力。
Speaker Role and Language Diarization for Analyzing Multilingual Interviews for Language Proficiency of Older Adults
- 基于Whisper的说话人角色与语言分割系统，自动提取受访者发言与语言使用模式。
- 受访人发言比例和目标语言使用率是语言能力评分的强预测因子。
- 仅用简单行为特征即可媲美语音嵌入，适合大规模自动化评估。
在基于多语访谈的语言能力自动评估仍属研究空白。本文构建基于Whisper的说话人角色与语言分离系统，自动提取老年群体多语访谈中的受访者发言，并分析其语言使用特征。实验表明，针对低资源且语系相关的印度语言，经语言适配的Whisper模型显著提升语言分离性能。统计分析显示，受访者发言占比与目标语言使用频率为语言能力评分的强预测因子。此外，仅用分离生成的行为特征即可达到与Whisper语音嵌入相当的评估效果，二者结合表现最优。重要的是，即使使用全自动化分离输出，语义分析与能力评估性能仍保持稳定，验证了以受访者为中心的对话分析在可扩展语言能力评估中的潜力。
原文摘要 · Abstract (English)
Automatic language proficiency assessment in the context of multilingual interview-based settings remains underexplored. In this work, we develop Whisper-based speaker-role and language diarization systems to automatically extract respondent speech and characterize language usage in multilingual interviews with older adults. We further investigate whether diarization-derived conversational and language-use behaviors can support downstream language proficiency assessment. Results show that language-adapted Whisper models substantially improve language diarization performance for lower-resource and linguistically related Indian languages. Statistical analyses reveal that respondent speech ratio and intended language usage are strong predictors of proficiency ratings. Furthermore, simple diarization-derived behavioral features achieve performance comparable to Whisper-based speech embeddings for proficiency prediction, while combining both yields the best results. Importantly, both the speech and language use statistical analyses and language proficiency prediction performance remain largely preserved when using fully automatic diarization outputs, demonstrating the potential of respondent-centric conversational analysis for scalable language proficiency assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。