提出HMamba模型，同时实现发音评分与错误定位。
Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss
- 设计层级选择性状态空间模型，融合发音评估与错误诊断。
- 在speechocean762数据集上，错误检测F1达63.85%。
- 引入解耦交叉熵损失，提升错误定位精度，适合语言学习系统开发。
现有计算机辅助发音训练（CAPT）系统通常将自动发音评估（APA）与发音错误检测与诊断（MDD）分开处理：前者提供多层级发音评分，后者聚焦于精准定位非母语者发音错误。然而，理想的CAPT系统应同时高效完成两项任务。为此，本文提出HMamba模型，首次实现APA与MDD的并行统一建模。同时，设计专用损失函数解耦交叉熵损失（deXent），针对MDD任务优化监督学习，显著提升错误检测能力。在speechocean762基准数据集上的实验证明，该方法在APA任务上表现优异；在MDD任务中相较强基线显著提升，取得63.85%的F1分数。代码已开源。
原文摘要 · Abstract (English)
Prior efforts in building computer-assisted pronunciation training (CAPT) systems often treat automatic pronunciation assessment (APA) and mispronunciation detection and diagnosis (MDD) as separate fronts: the former aims to provide multiple pronunciation aspect scores across diverse linguistic levels, while the latter focuses instead on pinpointing the precise phonetic pronunciation errors made by non-native language learners. However, it is generally expected that a full-fledged CAPT system should perform both functionalities simultaneously and efficiently. In response to this surging demand, we in this work first propose HMamba, a novel CAPT approach that seamlessly integrates APA and MDD tasks in parallel. In addition, we introduce a novel loss function, decoupled cross-entropy loss (deXent), specifically tailored for MDD to facilitate better-supervised learning for detecting mispronounced phones, thereby enhancing overall performance. A comprehensive set of empirical results on the speechocean762 benchmark dataset demonstrates the effectiveness of our approach on APA. Notably, our proposed approach also yields a considerable improvement in MDD performance over a strong baseline, achieving an F1-score of 63.85%. Our codes are made available at https://github.com/Fuann/hmamba
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。